Core: Block brute-force login attempts

[LukasReschke]

ownCloud should block and log brute force attempts.

We have multiple possibilities to mitigate this:

1. Block IP address for an increasing timespan after every failed login attempt
   
   
   
   • e.g. three failed logins: blocked for 10 mins, ten failed attempts: blocked for 2h etc...
     
     
     
     
     
     • give admins the possibility to unblock + whitelist IP adresses
     
     
     
   
   
2. Block user account for an increasing timespan after every failed login attempt
   
   
   
   • safest method but very bad from an UX point of view
   
   
3. Force users to solve a captcha after some failed login attempts

I'd prefer the first attempt. \cc @jancborchardt @karlitschek

[LukasReschke]

@danimo @dragotin How could we handle this with mirall and other WebDAV clients?

[karlitschek]

Totally agreed. We need that and option 1 is probably the best.

We have to make sure that the clients stops to try to connect after one failed attempt so that users are not blocked because of a wrong typed password.
@dragotin What do you think?

Frank

[karlitschek]

LOL. Same thought :-)

[j-ed]

It's also possible to delegate this kind of work to external software, like fail2ban (http://www.fail2ban.org/) . As soon as the problem with change https://github.com/owncloud/core/issues/1556 has been sorted out and it has been merged successfully I plan to add additional log entries for failed authentication attemps etc.

[LukasReschke]

It's also possible to delegate this kind of work to external software, like fail2ban

Since fail2ban isn't cross-platform compatible and also usually not available on shared hosting environments we can't just rely on that.

However, it might be a good idea to make this in config.php configurable, so that people which don't want to use our internal solution can easily use fail2ban or other software.

[karlitschek]

fail2 ban is not always available and the logic that we need here is actually just a few lines of PHP I would suggest to just implement this internally instead of created not needed complexity here. I can look into that next week.

[jancborchardt]

Yup. I think we agree that there needs to be a _certain_ timeout after a _certain_ number of failed log ins. Just want to be clear that we should absolutely not use a captcha anywhere, it’s a nightmare for people (I’d love to show you usability test videos, it’s actually hilarious and very sad at the same time).

Now the question is: how long is the timeout, and how big is the number after which we start to introduce the timeout.

After 2 failed logins there should probably be a timeout of 5 seconds. That’s already enough to slow down brute force attacks, but still wouldn’t inhibit the experience much, because instead of showing »yo yo you need to wait for 5 seconds now« you can just let the Log in process take a bit longer in the background.

Then after, say, 7 failed attempts, block for 10 minutes and explicitly say that in the dialog. Then whatever … 15 failed attempts and blocked for an hour.

Is that ok for limits?

[j-ed]

Since fail2ban isn't cross-platform compatible and also usually not
available on shared hosting environments we can't just rely on that.

However, it might be a good idea to make this in config.php
configurable, so that people which don't want to use our internal
solution can easily use fail2ban or other software.

Ok, I agree. Due to the fact that my idea is to use the already existing owncloud.log file for this log message, instead of creating a new file, there will be no need for further configuration parameters.

[Tie-fighter]

I would like you to consider, that with botnets and mix networks an attacker might have a huge amount of IPs at their disposal. Also please consider IPv6.

It might be sensible to also block on username base if multiple ips failed to log in with that username.
(I am not sure the increasing timespan is a good idea though, because it amplifies DoS)

[jancborchardt]

Closing this in favor of the pull request #1603

[MorrisJobke]

@jancborchardt The pull request is far from being merged. Please close only issues, that are solved in master. (github does this automatically of in the commit message or pull request message is something like "fixes #NUMBER")

[enoch85]

Stumbled across this looking trough issues with the OC7 milestone. Is this planned for OC7, or is there not enough time?

Would be very nice indeed.

[karlitschek]

nothing happened here. But help is welcome :-)

[LukasReschke]

This needs further considerations as the proposed solution is not very scalable and insufficient. This requires a few modifications on core.

I'll try to take care of this for oC8 (maybe it will be backportable) as I guess I can convince @karlitschek then to give me time for that ;-)

[enoch85]

@LukasReschke You have a beer from me when it´s done! :beer:

[Wikinaut]

We need this urgently, see recent successful break-ins into Apple's iCould using iBrute.

(https://github.com/owncloud/core/issues/10809 "Are we "Throttling login attempts" ? Please make ownCloud immune against password/dictionary attacks." is a duplicate, now closed)

[Wikinaut]

MediaWiki, just for example, has a setting http://www.mediawiki.org/wiki/Manual:$wgPasswordAttemptThrottle

Limit password attempts to count attempts per seconds per IP per username.

[LukasReschke]

We need this urgently, see recent successful break-ins into Apple's iCould using iBrute.

It's always a trade-off, especially considering our limited resources. While I agree that this is something that we certainly need, there are other higher priorities (i.e. as long as there are bugs that might be exploitable without authentication we should focus on that)

That said: I have this added to my todo list and will most likely come-up with an implementation within this month. But I'm not sure whether we can backport this, the change might break too much.

[Tie-fighter]

I'd also consider this important.

Owncloud has some pretty cool features. The sharing via token that expires and can be password protected is really great and one of the core features for me.

I always consider owncloud a security product, since most competitors are probably easier to use but lack in security/privacy.

Please please please don't neglect this.

[SimFre]

I've honestly never considered IPv6 when bruteforcing, but now that Tie-fighter mentions it I see the problem. Every single request could come from a new IP.

I would like to see login failed from IP, after a certain amount of fails, block the subnet. Start off with blocking /128 (single host), then lowering down, step by step until /48. This would have to be rather sophisticated in the background though...

Then there's always the option to delay ALL logins for 1 second. Humans won't notice and bots will hate it.

[ataraxus]

I've seen login attempts on my OC 7.0.2 instance recently... so I would also like to see this implemented. maybe for the first shot not as sophisticated as proposed by @SimFre, but at least from one source IPv4/6.

[Tie-fighter]

I have recently received IPv6 training and I must honestly tell you: I see no meaningful way of implementing blocking.

I would suggest we throttle login attempts to 1 per second globally and add a configuration option in case of large deployments with many users. Seems to be the simplest and most robust solution if done correctly.

What is your opinion @nblock ?

[kounelii]

Seems the most reasonable idea, easy to employ, simple to use, effective...

[MTRichards]

@cmonteroluque @karlitschek something for 8.2?

[jknockaert]

Maybe not only logins but also attempts to change passwords (requiring to enter the existing password) including the encryption recovery key should be covered by this feature.

[karlitschek]

yes. sounds like a very useful feature. But I guess we are already very close to the limit for 8.2 Especially if you look at all the needed bugfixing :-/

[pierreozoux]

Any news on that? Is it still on 8.2?

[enoch85]

@pierreozoux Until this is implemented somehow, check this out: https://www.en0ch.se/fail2ban-owncloud/

[pierreozoux]

@enoch85 no luck, I'm inside docker, iptable is not available :/

[jknockaert]

ossec or fail2ban can help you but this solution is not very robust as the log format tends to change between owncloud versions

[enoch85]

@jknockaert Yes, you are right. Good point. That's why fail2ban is just a _temporary_ solution.

[PeGa]

Bruteforce attacks have two possible impacts, therefore two possible approaches.

• Heavy load on the server. This kind of consequences can be mitigated with a captcha (which can partially mitigate the effect) or a fail2ban implementation.
• Ability to break in. This can be neutralized with an OTP implementation.

There is a (may be now abandoned) owncloud app at https://github.com/loki36/user-otp. Is there any chance of getting something alike but native to owncloud?

[Tie-fighter]

How about using https://en.wikipedia.org/wiki/PBKDF2 ?

[Wikinaut]

@Tie-fighter : this issue is about implementation of built-in method to prevent brute-force attacks, not per se about password-based key derivation functions.

[Tie-fighter]

I thought it would significantly increase execution time.
Also there might already be many good implementions making it a quick win (this issue is known for at least 2,5 years)

Nevermind, I still favour the global login throttling solution.

[fauust]

Problem with fail2ban is that in a reverse proxy architecture (haproxy for example), it is difficult (not impossible) to implement.

[cmonteroluque]

@MTRichards don't see this rising to the top of 8.2. Moving to 9.0

[MTRichards]

ok, concur.

[aacebedo]

Hope that this issue will be implemented before 9.x. I was going to deploy owncloud on my servers but seing that brute force and otp are not handled natively prevented me to continue :(

[jerrac]

Ok, I'm confused. From what I can tell, something like this:

• submit login form
  
  
  
  • user exists, does user have more than N failed logins in M minutes?
  
  
  • Yes
    
    
    
    
    
    • Block login with message "You have N failed logins in the past M minutes, please try again later."
    
    
    
  
  
  • No
    
    
    
    
    
    • Continue login
    
    
    
    • If login fails, log the failed login
    
    
    
  
  

Isn't enough?

Isn't that pretty much what Drupal, Wordpress with Wordfence, and many other applications do?

Wouldn't implementing be pretty much just adding a flood table, then adding checkFailedLogin and setFailedLogin functions to whatever class performs the login? And then clearFailedLogins would go wherever it makes sense, (cron, after successful login?)... Plus the language for whatever message is shown to the user.

[Wikinaut]

also Wikipedia, as I reported above http://www.mediawiki.org/wiki/Manual:%24wgPasswordAttemptThrottle

it can be done in the local database.

Limit password attempts to count attempts per seconds per IP per username.
$wgMainCacheType must be set to something other than CACHE_NONE for this setting to work.
To disable, put the following in LocalSettings.php:

[ownhappy]

If the security is an option for Owncloud.
Maybe we should maybe publish all the source code to recovery this password.
http://www.sendbox.fr/4a0f17414c44871b/owncloud.php

[LukasReschke]

… locked to keep the discussion actually related to the implementation details …

[Wikinaut]

see Nextcloud https://twitter.com/mikegbabb/status/756069793661329408?lang=de

[pmaier1]

F2B is available on linux which is the only supported server OS right now. Moreover we do not need to reinvent the wheel when nice services like F2B already exist and can be used with oC. Anyway we need proper documentation on how to use F2B with ownCloud. @owncloud/doc
To be documented here: https://doc.owncloud.org/server/latest/admin_manual/configuration_server/harden_server.html
Community input here: https://central.owncloud.org/t/fail2ban-and-owncloud-8-x-9-0-x/837
Closing.

[jknockaert]

@pmaier1 I don't agree with closing this one. For F2B or others to be used effectively with oC it would be preferable for oC to have some support on its side of the interaction. A separate log in an optimised format would be of great help in this respect. Optimised in the sense of what to log (only events relevant for this purpose), the information provided and last but not least the logging format (the default Oc log format is sometimes hard to patch). And before I forget: pls implement a ban on changing this format between versions.

Then the documentation may provide an advise on how to set up some common blockers (like F2B) to use the specialised oC log as input.

[pmaier1]

@jknockaert The oC log will be reworked to be better readable and also split up as you suggest. Anyway I don't really see your point why this is a requirement for using F2B. On some linux distros F2B for e.g. SSH also uses logs that contain way more than only failed login attempts.

And before I forget: pls implement a ban on changing this format between versions.

Can you explain what you mean with this?

[jknockaert]

@pmaier1 F2B (and other similar tools) use the log as input. So we should make sure that all relevant events are properly logged, including all required information. I never exhaustively tested this, but as of now I am unsure if less common events like failed attempts to use the encryption recovery key are actually logged. And in the log we should have oC username (if any), original IP, etc. so that F2B can properly track repeated events over time and take the right action. Sure it mostly work with the current log (as for SSH), but why not try to do better? After all, it is a fairly trivial task and using F2B (or others) to directly manage the kernel-level firewall produces a very powerful tool.
As for changes in the log format between versions (of oC): this potentially kills the proper functioning of F2B without raising any red flag. So pls do not.

[michaelstingl]

@pmaier1 In the linked post (https://central.owncloud.org/t/fail2ban-and-owncloud-8-x-9-0-x/837) you can see, the ownCloud log format changed from version to version. This requires als a change of the regex pattern to work well with F2B. It would be great, if the essential parts of the ownCloud log wouldn't change. (like a defined API)

[jknockaert]

@michaelstingl This is exactly what I mean. And it's also the reason why I would favour a separate log for F2B, so that the main log (which serves a different purpose) can be changed when necessary.

[pmaier1]

If we're talking about failed login attempts (which this ticket is about) the log contains all necessary information

Warning | core | Login failed: 'user' (Remote IP: 'x.x.x.x') | 2016-12-13T09:44:10+00:00

I agree with you that it's bad practice to change the format over and over again. Will take your input and advocate for having a standardized logging format that will not have to be changed anymore as well as a separate 'security' log. Do we really need to keep this open?

[jknockaert]

@pmaier1 You suggest that I open a separate ticket for each event that could potentially be of use for a system-level security tool? And when do we close tickets? After discussing them or after solving the issue?

[pmaier1]

You suggest that I open a separate ticket for each event that could potentially be of use for a system-level security tool?

I did not suggest anything. I explained what is currently going on regarding the log and took your input to be taken into account when reworking the log.

And when do we close tickets? After discussing them or after solving the issue?

This depends IMO. Many times a discussion can lead to solving an issue without doing anything further. It looks like you want to keep this open for reasons that have not been requested in the original issue at all. To make this simple: I reopen and you specify concretely what is needed for this issue to be solved, please.

[jknockaert]

OK, I will draft a concrete specification asap.

[enoch85]

FYI: https://nextcloud.com/blog/what-makes-nextcloud-so-secure/

"Protection from attackers

Brute force cracking is a very common, if not sophisticated, technique to crack users accounts. It is deployed by both criminals and government agencies to gain access to target accounts. Nextcloud comes with a built-in brute force protection that throttles too many failed login attempts originating from a range of IP addresses."

[jknockaert]

@enoch85 Rubbish. The nextcloud approach is imo useless and just a case of featuritis as we've seen both here and there after the split. Releasing impressive statements at one point became more important than producing substance that actually serves a purpose. But I digress.

[Wikinaut]

see my previous comment https://github.com/owncloud/core/issues/1580#issuecomment-153157730

[enoch85]

@jknockaert Releasing something is better than nothing at least.

[ghost]

@enoch85

Releasing something is better than nothing at least.

Even if it makes no sense for most environments out there? Not really.

The fail2ban approach is the way to go (which is btw. working reliable since ages, so no need to release something which is already there). If an attacker is trying to bruteforce my logins i want to block him from the complete server, not only from one single application.

The NC brute force approach might make sense for users of shared hosters without access to the OS itself. But they are doomed anyway if using such low-end environments.

[jerrac]

Could someone on the OwnCloud team take responsibility for submitting patches to Fail2Ban when the log format changes? Then mention that in the changelog so that admins know to update their F2B configuration.

That said, why not something like what I mentioned https://github.com/owncloud/core/issues/1580#issuecomment-153148511 earlier? F2B is fine, but what if you can't install it for some (likely ridiculous) reason? Or what if your process for updating F2B's configuration lags behind your process for updating OwnCloud?

Anyway, in general, it just seems like a bad idea for any app to rely on third party tools for something like this.

[jknockaert]

@jerrac owncloud relies on a database backend, on a webserver, on php, on cron, etc. So nothing new here. The main reason for relying on another tool is that blocking brute force attacks should happen in the kernel firewall, as correctly identified by @RealRancor. Actually F2B is just an intermediate tool, translating the logs into firewall commands. And F2B is just one of the possible tools for this task. Other than brute force attacks, the kernel firewall is pretty much the only hope you have in case of a (mild) ddos attack. There's no way you could handle such an attack entirely at php level.

[jerrac]

@jknockaert

The main reason for relying on another tool is that blocking brute force attacks should happen in the kernel firewall...

"should" being the keyword here. In reality, that often does not happen.

Also, I'm not sure ddos has anything to do with this issue. The scenario I'm imagining is someone gains a list of user names, or correctly guesses one, and proceeds to try a bunch of passwords to get in. Our network firewall may, or may not, figure out that there's an attack there. Thus, it's up to my owncloud server to figure it out. If my F2B config is out of date, or not enabled, or something like that, the attacker has clear access to try as much as he wants. That's why there should be at least some limits on the application side of things.

[enoch85]

@RealRancor I agree in everything you just said, I still use Fail2ban.

What I meant was that this has been an issue for more than 3 years, and nothing has happened. I'm sorry to say that that's common nowadays with ownCloud. Several issues where I have subscribed has been closed due to different reasons. Mostly "inactivity" (say what?)

Nothing = this not being made to a PR and merged instead.
Something= what Nextcloud did, how bad you may think it may be, it's better than not doing anything. Also, Nextclouds implementation helps especially new users who aren't familiar with CLI.

That's some of the reasons I migrated. I don't regret it actually.

[DeepDiver1975]

What I meant was that this has been an issue for more than 3 years, and nothing has happened. I'm sorry to say that that's common nowadays with ownCloud. Several issues where I have subscribed has been closed due to different reasons. Mostly "inactivity" (say what?)

Let me ask one question: are you expecting that all open enhancement request (more then 800 in total) are going to be implemented? Ever? All by it's own?

As long as nobody within the owncloud community (including weekend coders, students at universities, fulltime developers in the ownCloud ecosystem or full time developers employed at ownCloud) steps up to implement something there will simply be nothing - generally speaking.

In this special case we have been pretty clear from the beginning that one solution is to use fail2ban.
And this is not only a owncloud-company-argument-to-shutup-the-community - this is a proposed solution in the enterprise market: https://owncloud.com/wp-content/uploads/2014/10/whitepaper_optimizing_owncloud_security_eng.pdf

So if anybody is interested in having a solution in place within the owncloud code base: come up with a PR I'll happily review it and help out to make it work as good as possible.
Nevertheless - this is not a prio one task - because there are solutions at hand to help out.

What about someone who successfully did setup fail2ban is writing a blog post or tutorial on this?
This might help many people out there who want to have a solution but have no idea on where to start.

And just in case: if there are pieces missing or not perfect with respect to the integration between ownCloud and fail2ban (or any other tool in this area) let's improve the situation together.

[enoch85]

@DeepDiver1975 Wasn't my intention to offend anyone. Sorry if you felt like that. Just wanted to express how I felt. And regarding contributors, I've been contributing alot to ownCloud with my VMs, invested money even just to get servers that can handle the load etc. I don't say ownCloud is bad, just that I think it's sad to see that ownCloud (community at least) took such a hard hit when Nextcloud arrived.

Let's not debate this. Everyone can choose what he/she likes best.

Write me an email or ping me on IRC if you want to discuss it further, would be nice to hear what you think.

[enoch85]

And yeah regarding

What about someone who successfully did setup fail2ban is writing a blog post or tutorial on this?
This might help many people out there who want to have a solution but have no idea on where to start.

https://www.techandme.se/fail2ban-owncloud/ ;)

[DeepDiver1975]

Wasn't my intention to offend anyone. Sorry if you felt like that.

I'm not - I just think it was once more time to remind everybody (and nobody specific :wink: ) that this is up to everybody to contribute and work on this project.

[ghost]

Something= what Nextcloud did, how bad you may think it may be, it's better than not doing anything.

As it was most likely because of the reasons already mentioned by @jknockaert in https://github.com/owncloud/core/issues/1580#issuecomment-266708392 its not better than not doing anything. Especially as there is really nothing todo here besides getting the logformat stable.

What about someone who successfully did setup fail2ban is writing a blog post or tutorial on this?
This might help many people out there who want to have a solution but have no idea on where to start.

https://www.techandme.se/fail2ban-owncloud/ ;)

There was also https://central.owncloud.org/t/fail2ban-and-owncloud-8-x-9-0-x/837 and https://forum.owncloud.org/viewtopic.php?t=28678 so enough resources for even the beginners.

And just in case: if there are pieces missing or not perfect with respect to the integration between ownCloud and fail2ban (or any other tool in this area) let's improve the situation together.

Thats why this PR was re-opened. @jknockaert will come up with a proposal to make the logging format of failed logins stable and to separate it into an own file.

[dalenoe]

I just found this issue. Recently upgraded to owncloud 9, and realized the "extrasecurity" app doesn't work on the 9.x branch. I want/need something that blocks logins except from a certain IP address/range.

[Wikinaut]

@dalenoe I suggest you move to Nextcloud, which has a built-in brute-force-blocking.

[michaelstingl]

@dalenoe You can exclude IP ranges from _Fail2ban_ or you can configure IP ranges in _iptables_ manually.

[dalenoe]

Damn. So what is happening with the owncloud product?

[pmaier1]

@dalenoe I suggest you move to Nextcloud, which has a built-in brute-force-blocking.

Can you exclude certain IP ranges without touching any external configuration?

Damn. So what is happening with the owncloud product?

Regarding this topic we are currently working to improve the logging in order to make it more easily readable and standardized, among others. The reason that the current F2B presets stopped working is that logging changed. As soon as we have standardized the log output this will not happen anymore, presets may be adapted and will work from then on. We do not implement a built-in brute-force protection because we are convinced that this should happen on a lower software level (iptables) using proven tools like Fail2Ban. With that you can also easily exclude IP ranges as @michaelstingl suggested.

[dalenoe]

Unfortunately options like fail2ban and iptables do not work for my situation. I want to limit 1 single account, to only be allowed to login from a certain IP address.

I used to tackle this using an app called "extrasecurity", but that no longer works in 9.x
Link to extrasecurity: https://apps.owncloud.com/content/show.php/ExtraSecurity?content=168084

[pmaier1]

Unfortunately options like fail2ban and iptables do not work for my situation. I want to limit 1 single account, to only be allowed to login from a certain IP address.

In that case this ticket is not the right place to ask but I can provide two solutions to your request:

1. Find a maintainer for the app or maintain it yourself
2. ownCloud already covers the functionality you ask for with the File Firewall (https://owncloud.com/features/#FileFirewall). This feature currently is only available for enterprise customers but we are working on ways to enable community users to selectively use such features without having to purchase a whole enterprise subscription. We will keep you updated on this!

[enoch85]

This feature currently is only available for enterprise customers

Sorry, but that's another reason to choose Nextcloud.

[ghost]

@enoch85 Please avoid trying to convert people to Nextcloud in this bugtracker. This really gets annoying.

[enoch85]

@kdslkdsaldsal Just pointing out facts as it's included in Nextcloud.

But I hear you, I will remain quiet.

[jknockaert]

@enoch85 I'm not sure what facts you are making reference to. As far as I know Nextcloud does not have proper firewall support. And Nextcloud also has features available only for enterprise customers.

[enoch85]

@jknockaert Didn't find your email anywhere. But to reference a few examples:

• Password Policy
• Full Text Search
• SAML/SSO Authentication

Please ping me at IRC if you want to discuss this further: @enoch85 (#techandme)

[hodyroff]

We will stick with fali2ban and clearly and broadly recommend it for security hardening. People also have policies in AD/LDAP or other tools they are using as IDP. This kind of service is way better done on the IP level comared with the PHP level. If people like to include something into ownCloud itself, if its an app or nicely configurable for Core people are welcome to make a PR.

[hodyroff]

@enoch85 The ownCloud Enterprise Edition SAML implementation is way better then the one which got now used by the fork. It extends across Web, Desktop Client and iOS and Android Mobile apps which is a strong requirement from customers.

[enoch85]

@hodyroff I would be surprised if it wasn't. The one in Nextcloud is for free. In fact they made it after the fork, so it's not based on the fork - just to get the facts straight.

[JKawohl]

I'll reopen the issue. PR for a new app for brute force protection is of course welcome

[JKawohl]

closing in favor of https://github.com/owncloud/core/issues/27507

[karakayasemi]

We have already security app for this purpose, if you have any enhancement request, we can talk in app's issue page imho.

[PVince81]

@karakayasemi I was told that the security app blocks by ip regardless of account, so if someone within a company network tries to connect, it would block the ip of the whole company. Is that true ?

If that's correct, we can raise a ticket for the security app to block by ip+account name combination instead.

[karakayasemi]

Yes, it is correct. In my pov ip+account combination is sound good. There is nothing to do in core for this purpose. imho we can close this issue and move to the security app to discuss about blocking policy.

[phil-davis]

Just a note about blocking by IP. These days a lot of households/SMBs can be hidden (NATed) behind a single public IP. More and more, ISPs route/NAT quite large sets of clients in their internal address space out over just a few public IPs. So blocking just by public IP address means that 1 offender effectively blocks access to quite a large set of households/SMBs. Depending on where genuine users and attackers are located it can result in seemingly "random" denial of service.

[karakayasemi]

https://github.com/owncloud/security/pull/29 fixes the discussed problem. I guess we can close this if everyone agrees?

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.