Wstg: Publish to OWASP/www-project-web-security-testing-guide

[victoriadrake]

What would you like to happen?

Mirror the testing guide documents to the OWASP website repository to take advantage of inbound traffic and support SEO.

This repository will remain the source of truth for the WSTG project, while the mirror repo OWASP/www-project-web-security-testing-guide provides content to the WSTG project’s home on the owasp.org website (https://www2.owasp.org/www-project-web-security-testing-guide/).

• [x] Create www/ directory to separate website files for mirroring (_config.yml, index.md, info.md, etc)
• [x] Update info.md file to list links in sidebar and work with www—site-theme sidebar generation
• [x] Integrate documentation website theme (to be created) to utilize styling for documentation site (Jekyll)
• [x] Create GitHub Action to automatically push www/ and necessary files to OWASP/www-project-web-security-testing-guide

[kingthorin]

If we are gonna end-up duping to the www-project repo why not leave the www files over there?

[rejahrehim]

We can keep those www files in the www-project itself. We only need to generate and add a document folder based on the releases in this repo.

[victoriadrake]

A couple considerations: having the website files here as well would allow us to run tests (Actions can see if the site builds) before pushing to OWASP/www-project-web-security-testing-guide. We'd have things in one place, and OWASP/www-project-web-security-testing-guide becomes a no-touch mirror. At that point we're not really managing two repositories.

If we strongly prefer to keep www files separated, we can still do that and run tests in OWASP/www-project-web-security-testing-guide instead. It adds some complication to the site development process in the short term (generation files are in one repo and content is in another) but allows the site files to be separate.

[kingthorin]

It really isn't a huge deal either way, and we can always do a phased change as well. For whatever reason I just though minimizing change/deviation was best or perhaps easiest. (It was subconscious or knee jerk not deeply thought out.)

[kingthorin]

Food for thought:

• We might need versioned links for long term linking. ex: www2.owasp.org/www-project-web-security-testing-guide/wstg/v4/<some static destination or article>
• When we deem a version ready or published I think other projects (or people) might benefit from a simple md (or even better json/yaml) version of the test headings (to generate checklists, etc).

_{These came up in discussions with the zap team}

[rejahrehim]

Food for thought:

* We might need versioned links for long term linking. ex: `www2.owasp.org/www-project-web-security-testing-guide/wstg/v4/<some static destination or article>`

In that case, we have to do the migration of current v4 MD files to www-project as well?

* When we deem a version ready or published I think other projects (or people) might benefit from a simple md (or even better json/yaml) version of the test headings (to generate checklists, etc).

We can keep all versions in a separate folder inside www-project and wstg will have the builds in releases section too.

These came up in discussions with the zap team

[kingthorin]

In that case, we have to do the migration of current v4 MD files to www-project as well?

Well we can choose to start with v5 or whatever. (It's on us to decide)
We could have a "live" version too or something (that gets rebuilt on PR merge). Then when we decide to roll or publish at a particular point just push it to v<whatever> (in www-project) and not update it again? I dunno...

_{Edit: Actually unless we're going to cherry pick specific commits I think we're already too far along to publish v4 as such.... We've already removed lead-in/frontispiece v4 content, revised links, etc.}

We can keep all versions in a separate folder inside www-project and wstg will have the builds in releases section too.

I kinda figured wstg would always be 'latest' or 'live', but again we can decide what to keep where.

[victoriadrake]

We can keep all versions in a separate folder inside www-project and wstg will have the builds in releases section too.

I like the versioning idea for www-project, since it lets people easily browse previous versions. However, I think only the most current version should be served on the site, so we may want to tell Jekyll to ignore an "archives" folder or wherever the documents of previous versions live. Anyway, this is a longer-term concern. We first have to produce v5. :sweat_smile:

We could have a "live" version too or something (that gets rebuilt on PR merge). Then when we decide to roll or publish at a particular point just push it to v<whatever> (in www-project) and not update it again? I dunno...

This seems like the most straightforward process to me. For example, when v5 is complete, we'd push to a directory with the path OWASP/www-project-web-security-testing-guide/v5/. Our "live"" version is really the equivalent of a dev or nightly, so that could live at OWASP/www-project-web-security-testing-guide/dev/. What do you think?

I kinda figured wstg would always be 'latest' or 'live', but again we can decide what to keep where.

The shift from traditional publishing to GitHub is a bit of a change; the whole idea of versioning was a bigger deal when books had to be printed to be distributed. From a modern standpoint, docs are a living and continually updated thing, which is probably why you (and I) think of only using the "live" version. Having the updating stop at a particular version is beginning to make less sense, though it still is relevant to publishing.

[kingthorin]

This seems like the most straightforward process to me. For example, when v5 is complete, we'd push to a directory with the path OWASP/www-project-web-security-testing-guide/v5/. Our "live"" version is really the equivalent of a dev or nightly, so that could live at OWASP/www-project-web-security-testing-guide/dev/. What do you think?

Agreed!

The shift from traditional publishing to GitHub is a bit of a change ... though it still is relevant to publishing.

The thing I (we) were thinking here is that if other tools like ZAP or SKF want to link to content they need to be able to ensure it's there, so having versioned entries lets them do that without us having to handle redirects or mappings. (We can always apply a banner or something to outdated versions saying "Latest content here > <...../wstg/live>" [you get the idea]).

[ThunderSon]

As I was reading, I thought of this:
<domain>/wstg/<version>/<test>
What I would like to see happening is that we provide the URL <domain>/wstg/<test> and it would redirect to the latest version available.
All other versions still exist, but are not the default one. Let's say tool X is actually after a certain version's data, they can reference it, unless it's a heavy load on the website, and damages the SEO.
We can push what we have now as 4.1 and start preparing for minor releases, but we are not that much into actual versioning? We'll need to dig deeper into smaller releases then, and specify which tests cases we're going to modify and add. Let me know if that's something you'd be interested for us to do, then we can schedule some meetings to make it happen.

[kingthorin]

Yup I'm good with not having a designation (URL component) for the latest or live version.

Pushing toward a 4.1 release seems like a good move. We can put some placeholders in the into sections that need it just to be clear that it's not a full release but a partial step between v4 and the up-coming v5.

[victoriadrake]

As I was reading, I thought of this:
<domain>/wstg/<version>/<test>
What I would like to see happening is that we provide the URL <domain>/wstg/<test> and it would redirect to the latest version available.

How would the redirection be achieved?

Pushing toward a 4.1 release seems like a good move. We can put some placeholders in the into sections that need it just to be clear that it's not a full release but a partial step between v4 and the up-coming v5.

I’m on board for this. @ThunderSon would you set up a milestone for it? We might want to come up with a couple standards for minor releases.

[ThunderSon]

@rejahrehim @hblankenship Might any of you be aware of any technique that could allow us to achieve the above mentioned redirection using gh-pages? Open for other suggestions as well.