Misp: Seeking MISP Architecture and Scalability Insight

[github-germ]

Are there docs to review or a group of people to talk with that discuss architecting MISP for scalability over time, e.g. hardware, network, experience with db's other than MySQL, db distribution/mirroring, etc.?

As we continue grow our use, we are starting to look at scaling up the system to meet performance needs.

Thanks.

[adulau]

Good point. Maybe we need to work on a complete section in misp-book to explain how to scale with MySQL (or distributed compatible DB), redis and reverse proxies/load-balancer in front.

[github-germ]

That would be quite helpful. I would expect their are orgs out there with experience, of course including yours, that could share good and bad choices. Would there be a way to poll folks for their experiences? Not sure if you have a list of prominent MISP deployments.

[github-germ]

Hi, checking back to see if there are any plans to assist MISP folks who need to scale their architecture.

[github-germ]

Would also help to have some insight from organizations that have been using MISP for some time on how they monitor usage to determine when a MISP instance might be reaching performance load thresholds that should be addressed.

[github-germ]

Re @FloatingGhost -1

Curious why you'd vote this down? Thx...

[FloatingGhost]

I downvote people that spam my inbox with "Have you looked at this yet?" Hundreds of times to dissuade the behavior

[github-germ]

I downvote people that spam my inbox with "Have you looked at this yet?" Hundreds of times to dissuade the behavior

I'm learning the culture. Appreciate you taking the time to educate us.

I was concerned that you were down on my idea of the community sharing MISP Monitoring & Scalability. Hope some can share ideas as I'd imagine some experience on this front is out there.

Thanks!

[swedishmike]

I'm quite interested in this subject too - I'm currently, quite unsuccessfully so far ;), trying to create a more distributed Docker installation. External Redis instance, external MySQL/MariaDB instance and so on. Where I'm currently getting a bit stuck is to get some 'worker' nodes up and running for the scheduled tasks and so on.

In theory my distributed Docker thinking should apply for scaling on tin/VM's too so I'm happy to exchange ideas and experiences with others too.

[github-germ]

We've ordered new hosts to scale up our MISP infrastructure for 2019, and want to pursue a new architecture. If others have any wisdom to share on scaling up MISP across multiple servers to improve performance, that would be greatly appreciated.

[cvandeplas]

In this domain, please feel free to initiate pull-requests with proposed documentation for the MISP-book. This is truly appreciated!

[github-germ]

Seems like nobody has insight to contribute on scaling MISP to a multi-host architecture. We will be experimenting with this in 2020.

[iglocska]

Perhaps pinging @0xtf would help, his presentation @ misp summit:

https://www.youtube.com/watch?v=lUK3otmIcH4&list=PL0y--LUb0eDys9ttkVBcLA8X_DMv6I5rP&index=15&

[github-germ]

Perhaps pinging @0xtf would help, his presentation @ misp summit:

SUPER. Thx.

[0xtf]

Thanks @iglocska! Apologizes for the not great delivery @github-germ 😅 You can check https://github.com/0xtf/HAMISPA for notes and you can ping me anytime (Gitter, Twitter, etc) in case I can be of help.