Fec-cms: Create automated tests and add better error handling for USAJobs API calls

[johnnyporkchops]

Related to https://github.com/fecgov/fec-cms/pull/1885

Completion criteria:

• [ ] Add additional error handling and template language so that in the event that the USAJobs API is down or any of the queried endpoints are not returning data we are alerted to this and direct users to the USAJobs page (preferrably a USAJobs page populated with all of our open jobs). From @PaulClark2: "USAJOBS is unavailable. Please visit usajobs.gov for more information."
• [ ] Add error handling for the hiring-path get request which is currently not coded to check to check for no data or null being returned
• [ ] Write automated tests that we can use to verify that our error handling works and that out app always returns matches what is posted on USAJobs

Reported Error:
On 7/27/18, the https://www.fec.gov/about/careers/ page threw an error due to some problems retrieving data from the USA JOBS api. (Here's the error in the logs: https://logs.fr.cloud.gov/goto/b52ce59f67ed40363196d6ed47b70d68). One theory is that we couldn’t get the job codes back for a short time:
raise JSONDecodeError("Expecting value", s, err.value) from None

[lbeaufort]

In grooming, we discussed that better error handling is the highest priority.

[qqss88]

code/files related to this ticket:
fec-cms/fec/home/templates/partials/jobs.html
fec-cms/fec/home/templatetags/open_jobs.py

[qqss88]

need to play with usajob api

[qqss88]

add a little style to the error message, refer to this example:
https://isabelcastillo.com/error-info-messages-css

[qqss88]

made some changes in this back-end module fec/home/templatetags/open_jobs.py:

add 2 error messages

USAJOB_SEARCH_ERROR = "Error: https://data.usajobs.gov/api/Search is not accessible at this moment."
USAJOB_CODELIST_ACCESS_ERROR = "Error: https://data.usajobs.gov/api/codelist/hiringpaths is not accessible at this moment"
add 2 response status check:
{code}
if response.status_code != 200:
return {'error': USAJOB_SEARCH_ERROR}

#query usajobs API for list of all hiring-path codes
codes_response = requests.get(
    codes_url,
    headers=headers
)
if codes_response.status_code != 200:
    return {'error': USAJOB_CODELIST_ACCESS_ERROR}

{code}

[qqss88]

add error message display on the front end:

{% for j in jobData %}

{{ j.position_id }} , {{ j.position_title}}

• Open Period: {{ j.position_start_date }} - {{ j.position_end_date }}


• Open To: {{ j.open_to }}


• Grade: {{ j.job_grade }} - {{ j.low_grade }} - {{ j.high_grade }}

{% empty %}
{% if error %}

{{error}} Please check with usajobs.gov for details.

{% else %}

There are currently no open positions available. Please check back on this page or on USAJOBs for the latest FEC vacancy announcements.

{% endif %}
{% endfor %}

[qqss88]

still need to verify what kind of auto tests need to be done:

request1 on USAJOBS
request2 on FEC.gov

check request1 == request2 in terms of job list?

[qqss88]

discuss on whether or not we need some kind of caching on this job?

[qqss88]

still need to figure out how to local testing this ticket.

[qqss88]

will get some unit test and mock testing down first. Still need somebody to pair on the local testing.

[pkfec]

I can help @qqss88.

I have clone your fec-cms repo on my local. Let me know when you are available we can test your changes together.

[pkfec]

@qqss88 Here is the error msg i see on the Careers page, if this is what you are looking for?

[qqss88]

Here is some quick decisions based on out discussion(with John and Priya):

1. we only a error message for search query
2. no error reported if code list query failed
3. add a cached version(hardcoded version) of code list in case code list query failed.
4. if code is not available from either code list query or cached version, hirepath data will come from job json data
5. need unit test to test the logic.

[pkfec]

@qqss88 @JonellaCulmer @AmyKort

Below is the screenshot of the error on Careers page , when usajobs website goes down.

[JonellaCulmer]

@pkfec Thanks for sharing that screenshot.

@llienfec @AmyKort Has the draft language being proposed gone through a content review?
Also, I think we should link to USAJobs website.

@qqss88 Let's use the below warning message.
Here's references to where you can
Small size: https://fec-pattern-library.app.cloud.gov/components/detail/message-sizes.html
Warning icon: https://fec-pattern-library.app.cloud.gov/components/detail/message-alert.html

[johnnyporkchops]

@qqss88, great work! Since we have no PR yet, I will make some comments here:
I tested this locally by:
1) Feeding in a dummy json jobs-data file with a nonexistent HiringPath code and it works as expected-- Shows the literal HiringPath code from the response.
2) Changing the agency code to an agency that has no current jobs (example:AF2K) and works as expected -- Shows the "There a currently no open positions..." message
3) Change the USAJobs URL so that the response gets a 500 and works as expected --Get red error message.
4) Changed codes_response conditional to from !=200 to ==200 which loads the hardcoded CODE_LIST first and Got error -- But this was fixed by changing this typo from json.load(CODE_LIST) to json.loads(CODE_LIST)

I think we could move the hardcoded codelist to constants.py, what do you think?
Also, let me know if you need help with the front end styling for the error message.

[AmyKort]

The language in the error message looks good to me, @johnnyporkchops ! Thanks!

(@llienfec )

[JonellaCulmer]

@qqss88 Another thing I'd like to be checked is the typography used for the first couple of sentences. Those should be Gandhi serif.

cc: @johnnyporkchops

[qqss88]

@johnnyporkchops moving CODE_LIST to constants.py is a good idea. we can change the name to USAJOBS_CODE_LIST in that case. There are 2 constants.py in out code base, I think you mean this one:
./fec/fec/constants.py

right?

[johnnyporkchops]

@qqss88 , Yes, I think that ./fec/fec/constants.py would be the logical place.

[qqss88]

I think this ticket is ready to be wrapped up based on both testing and code review.