Pvlib-python: CEC 6-parameter coefficient generation

[cdeline]

SAM is able to extract the CEC parameters required for calcparams_desoto. This is done through the 'CEC Performance Model with User Entered Specifications' module model, and coefficients are automatically extracted given nameplate parameters Voc, Isc, Imp, Vmp and TempCoeff. The method is based on Aron Dobos' "An Improved Coefficient Calculator for the California Energy Commission 6 Parameter Photovoltaic Module Model ", 2012

Ideally we should be able to work with the SAM open source code, extract the bit that does the coefficient generation, and put it into a PVLib function that would allow users to run calcparams_desoto with any arbitrary module type. At the moment we are dependent on PV modules loaded into the SAM or CEC database.

Thank you!

[janinefreeman]

SAM solution routine is located at https://github.com/NREL/ssc/blob/develop/shared/6par_solve.h , function "solve_with_sanity_and_heuristics" . Additional dependencies on other files 6par_newton.h, 6par_search.h, 6par_jacobian.h, 6par_lu.h, 6par_gamma.h, all located in https://github.com/NREL/ssc/tree/develop/shared .

[mikofski]

I'd like to try to take this up sometime soon, maybe by the end of the year? Is anyone else working on it? I haven't seen a PR. Does anyone have any strong opinions? Is seems like there's a SAM implementation, is there a PVLIB-MATLAB version already? thx

[cwhanse]

I'm not working on it. If it is practical, it would be great to wrap the SAM method. NREL put a lot of effort into the heuristics for initial values and updating. We have a function in PVLib for MATLAB but it's a very different algorithm. For consistency with the parameter databases it would be better to use the SAM algorithm.

[adriesse]

Do I read between the lines that the two algorithms do not converge on the same parameters? Then it would be of great interest to have python implementations of each! Or perhaps even more important: documents describing the full algorithms along with every heuristic, assumption and constraint they implement...

[cwhanse]

@adriesse yes. Each model fitting algorithm I'm aware of, will produce different parameter values from the same data although the predicted IV curves may be similar.

[markcampanelli]

FWIW I am trying to establish a common/fair metric to compare different model fits in PVfit. In addition to choosing, say, which residual(s) to analyze, the effect of model discrepancy significantly affects such comparisons. PVfit’s orthogonal distance regression (ODR) fits to the single-diode model (SDM) often are worse than other algorithms’ fits in terms of terminal current residuals, but I have observed consistently that this is likely due to model discrepancy. One might claim that ODR is being more “honest” here, but I think it really means that the model should be improved. Note that PVfit’s ODR alternatively optimizes the residual of the sum of currents at the high-voltage diode node in the equivalent-circuit model, and it doesn’t make unfounded assumptions about error-free measurements in all data channels except the terminal current (which also has implications for parameter non-identifiability). I have seen plenty of evidence that ODR fits are “good” in the absence of significant model discrepancy, such as for some (but not all!) double-diode model (DDM) fits across various material systems.

[wholmgren]

If it is practical, it would be great to wrap the SAM method.

Two kinds of practicalities here:

1. Technical. I don't know how to do it but I am sure it's possible. I am ok with it so long as pvlib remains straightforward to install from source.
2. Legal. We would need assurance from NREL that it's ok for pvlib to distribute this SAM code under the terms of a MIT license rather than SAM's mixed MIT/GPL license.

[janinefreeman]

In the next 6-8 months, the NREL team will be updating our python wrapper for the SAM software development kit so that you can pip install the sam-sdk and call its routines in a much more native pythonic fashion. That might be a quick way to implement the CEC parameter generation to solve both the technical and legal practicalities Will mentions above, as well as the many file dependencies involved with that routine. The SDK is licensed separately from the SAM open source code under an MIT type license, so that distribution with pvlib would be ok, and once we have it set up as a python package, that should make it a non-issue to include with pvlib installation from source.

To @adriesse 's point, the publication associated with the method implemented in SAM is here: http://solarenergyengineering.asmedigitalcollection.asme.org/article.aspx?articleid=1458865

[wholmgren]

Thanks @janinefreeman that is great news!

[markcampanelli]

@janinefreeman Does NREL's web site no longer provide free preprints of Aron's paper?

[janinefreeman]

@thunderfish24 I'm not finding a copy of it in NREL's pubs database, but it looks like you may be able to get it here (didn't create an account to give it a try): https://www.osti.gov/biblio/1043759

[adriesse]

@thunderfish24 I sent you a preprint.

[cwhanse]

@janinefreeman how's the work going on the pip install sam-sdk? I saw a big push to PySAM, is this the package?

@mikofski @wholmgren initial outline of ivtools

Relevant to #587

[janinefreeman]

@cwhanse Yes! @dguittet (who you'll get to meet at PVPMC) just released the initial version on PyPi and Anaconda! We would love for you to test it out if you have a chance. https://github.com/NREL/pysam has more information.

[cwhanse]

Niether pip install nrel-pysam nor conda install -c nrel nrel-pysam worked, it looks like the package isn't available yet?

[dguittet]

It's available for Python 3.5 and above. What errors are you seeing?

[cwhanse]

Proxy issues on my end, must have been some OS update I didn't see.

[dguittet]

Thanks for clarification.

[cwhanse]

Installed. My test case using 6parsolve is working. Nice.

[dguittet]

Thanks for checking! If you don't mind, if you are using PyCharm, do you get autocompletion suggestions like in this picture?

[cwhanse]

I'm using Spyder and yes, autocompletion is available.

[StefanGerber]

Good day. @cwhanse, would you be able to share your 6parsolve testcase?
I am finding it hard to setup the pvsamv1 module and an example would be of great help. Thank you

[dguittet]

@Stev0198 Thanks for the question. I've created an Issue on PySAM with examples and for further discussion.

@cwhanse Thanks for testing!

[cwhanse]

@Stev0198 see fit_cec_with_sam function in #718

[cwhanse]

@wholmgren @mikofski looking for ideas for testing fit_cec_with_sam. The function relies on PySAM which is a wrapper around SAM. Unless we install SAM in travis, we can't actually run fit_cec_with_sam.

[wholmgren]

You’d need to add it to the ci/requirements* files. Will be easiest to do that once the azure pr is merged. Go ahead with that if you’d like.

[cwhanse]

"it" = PySAM, or also SAM?

[wholmgren]

The PySAM installation instructions give me the impression that the PySAM packages on PyPI and conda include whatever is needed from SAM. If that's not the case, then I don't know what to do. If pypi wheels are not enough and we need to use the conda packages, then it would really be helpful if NREL would put the package on the conda-forge channel instead of their own channel.

[cwhanse]

The C code for the SAM function used here (6parsolve) is not included in PySAM that I see. @dguittet would appreciate your advice here.

[dguittet]

@cwhanse Downloading the wheel from PyPi will provide the code that's necessary to run 6parsolve like this:

datadict = {"tech_model": '6parsolve' ,'celltype': 'multiSi', 'Vmp': 30, 'Imp': 6, 'Voc': 37, 'Isc': 7, 'alpha_isc':0.004, 'beta_voc':-0.11, 'gamma_pmp':-0.41, 'Nser': 60, 'Tref':25, 'financial_model':'none'}
from PySAM.PySSC import *
ssc_sim_from_dict(datadict)
# {'celltype': 'multiSi', 'Vmp': 30.0, 'Imp': 6.0, 'Voc': 37.0, 'Isc': 7.0, 'alpha_isc': 0.004, 'beta_voc': -0.11, 'gamma_pmp': -0.41, 'Nser': 60.0, 'Tref': 25.0, 'a': 1.5672527775344784, 'Il': 7.062310805915052, 'Io': 3.458041948093099e-10, 'Rs': 0.3778517943111882, 'Rsh': 42.44789603827558, 'Adj': 25.238671555498257, 'tech_model': '6parsolve', 'financial_model': 'none', 'cmod_success': 1}

The PyPi package includes libssc.so (or .dll/.dylib) whose api is unchanged, so given the PySAM.PySSC import, all Python scripts using the PySSC wrapper will work as they did without PySAM. ssc_sim_from_dict is an new function in PySAM.PySSC that works with the original PySSC class.

I am in the process of adding functions which are done outside of a technology or financial compute module, such as the cmod 6parsolve or the SAM GUI equation pv_array_size in line 6339 of PV System Design.txt, to PySAM such that these functions are easier to use and their outputs merge with the PySAM classes.

@wholmgren We build PyPi packages with bdist_wheel and haven't had the chance to create conda recipes, which is why it's not available on conda-forge.

I will take a look at fit_cec_with_sam.

[wholmgren]

If the wheel works then no need to worry about conda channels/recipes. @cwhanse you should be able to add nrel-pysam to the pip section of the python 3.5, 3.6, and 3.7 builds.

[cwhanse]

@dguittet pip install pysam fails on Windows 10.
Appears to be a path issue, the error I get is

  ERROR: '.' is not recognized as an internal or external command,
operable program or batch file.

[cwhanse]

the correct comment pip install nrel-pysam works. thanks @dguittet

[adriesse]

@dguittet Should I be able to install from a cloned PySAM repository in editable mode? i.e. pip install -e .

[dguittet]

@adriesse Yes, you should from the PySAM directory as pip install -e .

In the PySAM directory, python setup.py develop also works.

python setup.py test will activate all the tests.

[adriesse]

@dguittet This fails for me, presumably because I don't have SAM installed--although from the foregoing discussion I thought that wasn't necessary. I wonder whether I should open an issue?

[dguittet]

@adriesse You don't need SAM installed. However, the package you download from the pip registry will contain binaries (.dll or .so) that are specific to your computer platform & python version. When you download the PySAM repo, these binaries are not included, and must be generated following the instructions for Building libSAM_api. Hope that helps. If you run into problems, do please open an issue!

Edited to add:

The PySAM package is mostly binary, consisting of Python C Extensions for each ssc compute_module. The only Python code in PySAM is files/PySSC.py. From my limited understanding of the use case of the editable mode, it is to keep Python source code in a separate location for development. However, any changes to C Extensions will require rebuilding. I would guess that if you're not planning to make changes to the C code in PySAM or the included PySSC.py, a regular pip install will be fine. If you are, then following the build process above would be necessary for local testing.

[adriesse]

@dguittet I admit I am starting to have some doubts about proceeding... I was hoping keep on the bleeding edge of updates and maybe be in a position to do a PR at some point--which could just be related to examples or documentation. I was not planning to rebuild from C code. Technically, I think I'm just missing the environment variables to point to those pre-built files, so I may give it one more try.

[cwhanse]

@adriesse You don't need SAM installed.

Does having SAM installed avoid having to build the binaries?

[dguittet]

@adriesse I see, thanks for the explanation. Building the ssc and SAM_api libraries needs to be done outside of PySAM, but if you're not modifying anything in the ssc or sam repo, this only needs to be done once.

Since you mention examples or documentation, that would be very helpful and interesting to boot. There could definitely be an examples folder to which you can push changes without necessarily having to build PySAM-- you simply install PySAM from pip and use that to run those example Python scripts. The documentation are .rst files that are auto generated and rely primarily on auto-doc from Sphinx. I'd be very interested in combining more detailed, customized documentation into these skeletons. Again, these changes would not require building PySAM.

The only caveat is that the version of PySAM you have downloaded from pip must be the latest release (currently 1.2.1) and that any of these examples or documentation changes would still be valid for the next development release (will be 1.2.2 but being developed currently as 1.2.dev1). As a recent example, if a group name or variable name changes between versions, that will break any Python code that uses those variables. Unfortunately since we're still ironing out the interface of PySAM, it's possible that things may change from one version to the next and that's when you'd have to build the binaries, run setup.py, and run tests to make sure any Python code using the underlying C Extensions is still valid.

I imagine the possibility of adding custom python functions to the PySAM classes would be desirable also. I've just opened a new issue for Python layer for customizing PySAM Classes.

Regarding the build process, I'm the only one who has tried it out so far... please do contact me if you run into issues as any other set of eyes would be greatly appreciated. There's still a lot of documentation and protocol-creation that needs to be done, and any current issues provide future answers, so thanks for testing it out.

@cwhanse SAM only comes with the ssc binary, and doesn't have the SAM_api, and each of the PySAM class binaries. However, if you downloaded PySAM from pip and only make changes to the Python code, described above, provided the variables that the Python code calls are still valid in the development version, you can use the downloaded PySAM package ( with binaries found in the site-packages/PySAM directory) to do development in the PySAM repo.

[mikofski]

@dguittet one potential solution would be to make a conda package. Since conda is an env spec (not just python), it can include things (not python) like compiler tool chains and/or binary packages. Eg:

1. you could create separate conda packages for libssc, etc., that can be distributed as binaries and have the build instructions in the conda recipe, then list them as requirements for pysam-nrel
2. Or if it's unlikely for any other application to use libssc, etc, then you could distribute pysam-nrel as a conda package, and specify the toolset

The reason this works is because conda has packages with compilers and build tools like cmake for every platform

[mikofski]

... the latest release (currently 1.2.1) and that any of these examples or documentation changes would still be valid for the next development release (will be 1.2.2 but being developed currently as 1.2.dev1). As a recent example, if a group name or variable name changes ...

FYI if you follow semantic versioning then I believe it's considered okay to make breaking api changes while the version is still zero, _eg_: 0.x.y, but then the convention is to limit breaking api changes after version 1.x.y has been released.

Since it's too late this time, maybe just something to consider for future projects. And there are other conventions that differ from SemVer, and of course I may be misinterpreting the conventions.

Thanks!

[dguittet]

@mikofski That's a good idea! I find myself and others using conda over venv more. I haven't spent any time with conda recipes, but if it could automate the build process then that would be a plus.

Regarding semantic versioning, you bring up a good point. I did break the convention going between 1.2 and 1.1 unfortunately. Next release will have to be 2.0.