Juice-shop: CTF-mode

[bkimminich]

_As a_ security class lecturer
_I want_ to be able to run Juice Shop in CTF-mode
_so that_ my students can compete for solving the most challenges fastest

Architecture Requirements

• Decouple Challenges table from shop data model _not needed, see ideas below_
• Add a _CTF-server_ to manage all students and track their progress (this must actually be _securely implemented_)
• Ensure recoverability of user sessions after individual Juice Shop instance crashed _continue codes offer that already_
• Change single-user variant to also run as _CTF-server_ + single Juice Shop instance (to harvest the recoverability benefits also for single users) _not needed, separate project for juice-shop-ctf-server instead_

[wurstbrot]

A point system would improve it. The first one who solves a challenge should get more points than the second one (look at OWASP Security Shepherd for example). Besides of that, the 1-3 Stars weight should be used.

Ensure recoverability of user sessions after individual Juice Shop instance crashed

This might be reached by synchronising challenge information from server.

As a student I want to choose my name or a nickname for the overview.

[bkimminich]

_As a_ Student
_I want_ to see the current point modifier per challenge on the score board
_so that_ I know what challenges to tackle first for maximum bonus points

[bkimminich]

Kind of a downer for the otherwise nice CTF idea: Juice Shop is not meant to have exchangeable/dynamic challenges, so you can basically cheat yourself through it in no time by checking the test suites, source code or external coverage like podcasts, conference videos etc.

Would still make a nice addition, to be used in classroom trainings for additional _gamification_. It's just useless as a _competitive CTF target_ on conferences etc.

[tghosth]

Hi @bkimminich, I was thinking about this.

I haven't dug in to the detail (because I still want to try that challenge) but I understand that continue codes are generated using a salt. As a hacky short-term measure for training gamification would it be possible to allow the following work flow:

1. Generate unique salts for each user in a way that still doesn't allow trivial reverse engineering of continue codes.
2. Let people use their own copy of the app.

3. Create a simple scoreboard app where:

   • Someone enters in the list of people and their unique salt
   • Participants can continually enter their name and their current continue code (or even embed an identifier in the continue code)
   • The app will use the continue code and user identifier (and associated salt) to update the scoreboard.

[bkimminich]

@tghosth: Your idea doesn't sound too hacky to me! 👍

• People would register with a central ctf-server with a chosen username. The ctf-server returns a numeric unique player ID in the response.
• Instead of using the continue code itself, each application instance - whenever a challenge is solved - could hash an array of [uniquePlayerId, challengeId] and send it to the ctf-server.
• The ctf-server could redraw its global score board via socket.io in realtime.
• Should someone crash his application. they can:
  
  
  
  1. restart their application
  
  
  2. register again at the ctf-server
  
  
  3. then use their local continue code to restore their progess
  
  
  4. which will trigger all solved challenges, and the corresponding calls to the ctf-server
  
  
• The ctf-server should remove "dead" players periodically from the global score board. Maybe by just expecting a ping from each player every few minutes?

Making this as tamper-proof as possible would need some extra thoughts, but the communication process could be done in this rather simple way.

[tghosth]

thanks :)

Based on your post, I have marked some areas which I could be enhancements rather than requirements for a first basic version, also added a couple of notes

• Instead of using the continue code itself, each application instance - whenever a challenge is solved - could hash an array of [uniquePlayerId, challengeId] and send it to the ctf-server.

So I think that the basic version could allow the player to manually paste the code and the autosending could be a later enhancement.

Also, if the app instance produces a hash, will the server have to brute-force it to discover which user performed which challenge? That is why I thought that a differently salted challenge token would be a good method.

• The ctf-server could redraw its global score board via socket.io in realtime.

Could be an enhancement

• The ctf-server should remove "dead" players periodically from the global score board. Maybe by just expecting a ping from each player every few minutes?

Could be an enhancement.

[tghosth]

Also:

• Enhancement: The scoreboard app could allow the training organiser to decide what points modifiers should be applied to each challenge and automatically calculate scores taking that into account.

Again, this feature will not help for competitive CTF but I think this app is perfect for a training type scenario because it is the only vulnerable app I have seen which:

• includes progress tracking and feedback on success
• is a realistic application
• Is based on a very modern and relevant framework

[bkimminich]

So I think that the basic version could allow the player to manually paste the code and the autosending could be a later enhancement.

I assume that the effort for manual challenge synchronization would be not much less than implementing it automatically right away. The application's server side already sends the push notifications on every solved challenge (to its own client) so it would basically just be another web call to the ctf-server that needs to be added. The rest of the implementation effort is the new ctf-server anyway.

Also, if the app instance produces a hash, will the server have to brute-force it to discover which user performed which challenge? That is why I thought that a differently salted challenge token would be a good method.

I guess we have similar solution in mind, just not exactly. This is where meeting in front of a whiteboard would be nice... 😁

The ctf-server shouldn't have to brute force anything. There should be an agreed-upon secret (e.g. salt for hashids) between application and ctf-server. This secret must never get to the application client, i.e. the browser, because then cheating would be way too easy. But it cannot be completely prevented, so effort in "security" should remain reasonable.

An even simpler alternative to the whole hashing idea could be this:

1. The player registers a username at the ctf-server.
2. The ctf-server creates a unique long string from the username and setup up an express-route dynamically, e.g. /webhooks/{connected-application-id}/challenge-solved/{challenge-id}
3. When a challenge is solved, this URL is called and the ctf-server immediately knows who solved what challenge was solved by whom

I have the feeling that the hashing would be more obscurity than security, so maybe this is the easiest to start with?

it is the only vulnerable app I have seen which includes progress tracking and feedback on success

Actually the score board idea I copied from the BodgeIt Store but the instant success notifications I never saw anywhere else before.

[bkimminich]

I've created a new repository https://github.com/bkimminich/juice-shop-ctf-server for the ctf-server part. Will populate it with Readme and the minimum basics needed for testing etc.

It would be fastest to just use the same tech stack as in juice-shop itself, but it could also be a chance to use the latest versions of those libraries. That might make the migration tasks (#165, #164 and #167) a bit easier later.

[bkimminich]

After learning about https://github.com/facebook/fbctf at German OWASP Day _and_ hearing that there are parallel activities going on to set this up with Juice Shop, I cleared the https://github.com/bkimminich/juice-shop-ctf repository to hopefully receive that solution, maybe as a 🎁 from 🎅 ... 😁

[tghosth]

Cool :) is their progress in a public github repository?

[bkimminich]

In case the Facebook framework turns out to be too unstable, here are some potential alternatives: https://github.com/apsdehal/awesome-ctf#platforms

[tghosth]

Hi @bkimminich,

To me it looks like the Facebook platform would be overkill (and complex to configure) for Juiceshop.

However, I had a look at https://demo.ctfd.io/ and it seems much more straightforward. I think the only thing that would need to be done in juiceshop would be to add a unique flag for each challenge and then only show it when the challenge has been completed.

Obviously storing the flag plaintext in the database would be a bad idea but potentially you could use an HMAC of the challenge details using a key stored in the code, it doesn't need to be complicated.

Is that something you could add? I could the look at creating instructions or a drop in file for configuring CTFD appropriately.

Unless of course you are aware of any progress on using FBCTF ?

[bkimminich]

The FB variant is ongoing. Commits will come in a few weeks, I hope. The idea there was to have a Docker-cluster of CTF-server, load balancer and several Juice Shop instances. Having an alternative setup with the CTFd - which already comes with it's own docker-compose file - why not? But even just a distinct config file plus instructions how to use it with Juice Shop would be great!

Displaying a challenge solution code would be required for both variants, so I'll open a separate issue for that subtask. I can implement this pretty quickly, I suppose.

[bkimminich]

@tghosth, the challenge notifications now contain also a flag field! Feel free to use it! 🥇
(Please use branch develop)

[tghosth]

cheers @bkimminich

What will be the best way of bulk extracting the challenge details and flags in order to load them into the CTF system?

Currently, it looks like the easiest way to enter them will be using SQL insert queries. The insert queries will need to be of the form:

insert into challenges (

id, 
name, 
description, 
value, 
category, 
flags, 
hidden

) values (

<<ID from juiceshop>>, 
"<<name from juiceshop>>", 
"<<description from juiceshop>>", 
<<to discuss value>>, 
"<<category based on difficulty juiceshop>>", 
"[{""flag"": ""<<flag from juiceshop>>"", ""type"": 0}]",
0

)

For example:

insert into challenges (

id, 
name, 
description, 
value, 
category, 
flags, 
hidden

) values (

1, 
"scoreBoard", 
"Find the carefully hidden 'Score Board' page.", 
1, 
"Difficulty Level 1", 
"[{""flag"": ""<<flag from juiceshop>>"", ""type"": 0}]",
0

)

My thought is that having a dynamic way of generating these insert statements would be best so as to account for new challenges being added.

What are your thoughts?

[bkimminich]

Doing this automatically would be _awesome_! Not only for new challenges but it would also allow to change the HMAC key from time to time (or just pick one randomly when the whole CTF environment fires up). This would mitigate some cheating options! :+1:

The categories would of course be nicer to have by vulnerability type (e.g. XSS 100, SQLI 200 etc.) but then the simple 100xDifficulty formula won't work any more, right? We might need to put some thought into that system, but I honestly didn't look into how CTFd handles this, yet.

Right now the HMAC flag per challenge is generated on the fly with a fixed key. But this is all handled within the server w/o any exposition via API or client-side code. It is only exposed via Socket.IO when a challenge is solved. As my end-to-end tests solve all challenges anyway, they could technically do that with a 3rd party tool listening on that websocket as well. That way you could harvest all the current flag codes before launching the CTFd. Too crazy idea? :grimacing:

[bkimminich]

I suppose it would help to have a proper category field in the challenge table, right? I did a classification anyway for the eBook, so this could simply be used for CTF as well:

[bkimminich]

"Finding the Score Board" technically belongs into "Information Leakage" - it is just its own section in the book to act as a starting point for the readers.

[tghosth]

• Regarding categories, I agree that splitting by vulnerability type is more elegant.
• Regarding flags, please see my comment on issue 260
• Regarding scoring, I think CTFd just uses a static score value. I would suggest that the CTF setup script uses the following default point system: 1=10, 2=15, 3=20, 4=25, 5*=30 but I am open to suggestions :)
• Regarding descriptions, these will need to include the difficulty level but that should not be hard to do in the CTF setup script.

[bkimminich]

@tghosth: /api/Challenges not also has the category in its JSON response. Also you can access the ctf.key in the root folder of the juice-shop repo (not via the app or API, obviously) to generate the flags on your own. The JavaScript code I use internally you find in https://github.com/bkimminich/juice-shop/blob/develop/lib/utils.js#L79 ff.

[tghosth]

hi @bkimminich, I have written something quick and dirty which will build the INSERT statements automatically.

Right now, it is just an ugly HTML page which pulls the challenges API from heroku, the ctf.key from github (both changeable) and uses the SHA code directly from github,

It outputs a bunch of INSERT statements which can then be used to populate the challenge database of CTFd.

Do you think that is sufficient? It is relatively automated.

P.S. Thanks for putting the server side work together for this :)

[bkimminich]

@tghosth, that sounds amazing! Do you want to submit a PR to the juice-shop-ctf repo for this? If you could add a few lines to the README how to set it up, that'd be awesome. Or I can give you commit rights to that repo, if you prefer.

[tghosth]

Created a PR for this. Like I said, it is very ugly but it should do the job without trying to be too clever.

[bkimminich]

@tghosth, It's not _ugly_, it's an MVP! :grin:

Thank you very much! I'll test it this weekend! (Email me your post address and shirt size and I'll send you one of our official polo shirts!)

[tghosth]

It's not ugly, it's an MVP! 😁

Ha ha, I will remember that :rofl:

I'll send you one of our official polo shirts

Thanks :) Emailed your OWASP account.

[bkimminich]

Just tried it, and the CTFd setup works. Even on my Windows machine (as long as you remember to forward ports in VirtualBox...) - So, huge thanks again, @tghosth!

(Closing this issue as the feature is technically implemented. I added a _Coming soon..._ section for the FBCTF setup in https://github.com/bkimminich/juice-shop-ctf/blob/master/README.md)

[lock[bot]]

This thread has been automatically locked because it has not had recent activity after it was closed. :lock: Please open a new issue for regressions or related bugs.