Products.cmfplone: Simple registry.xml entry leads to complete memory allocation/memory bomb

[zopyx]

Plone 5.1a.

I added the following registry.xml record to the almost empty registry.xml

https://github.com/xml-director/xmldirector.plonecore/blob/master/xmldirector/plonecore/profiles/common/registry.xml

<record name="plone.displayed_types" interface="Products.CMFPlone.interfaces.controlpanel.INavigationSchema" field="displayed_types">
    <field type="plone.registry.field.Tuple">
      <default>
        <element>Image</element>
        <element>File</element>
        <element>Link</element>
        <element>News Item</element>
        <element>Folder</element>
        <element>Document</element>
        <element>Event</element>
      </default>
      <description xmlns:ns0="http://xml.zope.org/namespaces/i18n" ns0:domain="plone" ns0:translate="">The content types that should be shown in the navigation and site map.</description>
      <missing_value/>
      <required>False</required>
      <title xmlns:ns0="http://xml.zope.org/namespaces/i18n" ns0:domain="plone" ns0:translate="">Displayed content types</title>
      <value_type type="plone.registry.field.Choice">
        <vocabulary>plone.app.vocabularies.ReallyUserFriendlyTypes</vocabulary>
      </value_type>
    </field>
    <value>
      <element>xmldirector.plonecore.connector</element>
      <element>Event</element>
      <element>File</element>
      <element>Folder</element>
      <element>Image</element>
      <element>Link</element>
      <element>News Item</element>
      <element>Document</element>
    </value>
  </record>

Running the related import step caused an immediate eating of all available RAM within seconds (machine has 64 GB). Removing the record and re-running the import step works without issues...weird.

[gyst]

We've experienced similar memory bombs in running GS steps when we were upgrading ploneintranet from Plone 5.0.2 to 5.0.4 - that suggests it's caused by something introduced after 5.0.2. @pbauer worked around that somehow. Sorry for not investigating deeper and filing a bug at the time.

[pbauer]

In our case it was legacy-style registration of js- or css-resources (using jsregistry.xml or jsregistry.xml). That triggered the cooking of the legacy-bundle with Products.CMFPlone.resources.browser.cook.cookWhenChangingSettings. Each resource triggers a subrequest with response = subrequest(siteUrl + '/' + resource.js). Each subrequest ate about 3GB of memory. I did not figure out why and simply removed the obsolete legacy-registration.

[gyst]

IIRC in our case this was related to registering legacy CSS/JS resources. We didn't actually need those and removing them avoided the memory bomb. But I've also seen a mitigation of memory explosion by not loading a specific profile that had no such respurces to begin with. So mixed signals.

[gyst]

Oops synchronicity :-)

[thet]

The subrequest memory leak bug was introduced with some CSRF related fixes. We experienced it with plone.app.mosaic, where rendering around 10 tiles bloated the memory beyond 2GB per request.
It was fixed here: https://github.com/plone/plone.subrequest/pull/8

[hvelarde]

guys, IMO, removing the legacy-bundle in Plone 5.1 is not an option given that it has not been working until today in Plone 5.0.x (it will only work after Plone 5.0.5 has been released).

this kind of problems have led to a slow adoption of Plone 5 out there as is hard to make add-ons compatible with it and without add-ons Plone is not very useful for most people.

please try to fix the bug and do not remove the legacy bundle until Plone 5.2, at least.

CC @plone/framework-team

[zopyx]

The legacy bundle can not be removed until there is a working resource registry implementation in place that is stable, usable and covering all usecase. Also keep in mind that Plone 4.3 will be around for a long time - like it or not. But in particular big installations are currently moving forward to Plone 4.X and will remain on Plone 4 for the time being. We are speaking here of universities running several hundred Plone installations....so many third-party add-ons are supposed to work both on Plone 4.3 and Plone 5.X. From the integrators point it does not make sense to cut off functionality and replace it with non-working functionality.

[hvelarde]

@thet thank you very much for pointing this out; I tested the fix in one production server and it fixes the memory leak I was having after upgrading to Plone 4.3.9.

I suggest to make ASAP a new release (1.6.12) of plone.subrequest in order to make things easier tor everybody.

[hvelarde]

@zopyx could you please confirm that this problem is fixed using the new plone.subrequest 1.7.0?

[zopyx]

Will do in a few days

Sent from a mobile device

Am 05.05.2016 um 19:39 schrieb Héctor Velarde [email protected]:

@zopyx could you plese confirm that this problem is fixed using the new plone.request 1.7.0?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub

[davilima6]

I confirm the issue in 5.1a1 and that upgrading to p.subrequest to 1.7.0 fixes it.