Products.cmfplone: copy & paste BROKEN

[iham]

BUG/PROBLEM REPORT (OR OTHER COMMON ISSUE)

What I did:

after creating the plone instance i add a content type with a richtext field (document, event, news item - does not matter which type in particular).
inside the richtext field i reference an existing image (or upload a new one - does not matter which way the image is created).
after saving this i want to copy that object and click "actions" -> "copy".
now i navigate to the target place where i want to paste that content object and click "actions" -> "paste".

the same happens during tests i wrote using:

# create image, doc and richtext
image = api.content.create(portal, type='Image', file
doc = api.content.create(portal, type='Document', title='doc')
doc.text = RichTextValue(
    '<p><img  src="{portal}/resolveuid/{uid}/@@images/image/large" class="image-inline" data-linktype="image" data-scale="large" data-val="{uid}" data-mce-src="{portal}/resolveuid/{uid}/@@images/image/large" data-mce-selected="1"></p>'.format(portal=u'..', uid=image.UID(),
    'text/html', 'text/x-html-safe')

# now lets update the richtext and linkintegrity
notify(ObjectModifiedEvent(doc))

# prepare a target to paste to
target = api.content.create(portal, type='Folder', title='test')
api.content.copy(doc, target)

What I expect to happen:

the object is pasted and displayed properly.
the test works.

What actually happened:

Traceback (innermost last):
  Module ZServer.ZPublisher.Publish, line 144, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module Products.PDBDebugMode.runcall, line 68, in pdb_runcall
  Module ZServer.ZPublisher.Publish, line 44, in call_object
  Module plone.app.content.browser.actions, line 252, in __call__
  Module plone.app.content.browser.actions, line 315, in do_action
  Module plone.app.content.browser.actions, line 221, in do_redirect
IntIdMissingError: None

What version of Plone/ Addons I am using:

vanilla plone 5.2rc2 with python 2.7
no addons

[ale-rt]

Hi @iham, can you try if this patch solves your issue?
https://github.com/plone/five.intid/pull/8/files

[iham]

i will try it and give you feedback.
thanks for reacting that fast @ale-rt !

[iham]

@ale-rt sadly no effect. same error as before

[ale-rt]

Thanks for trying and sorry that was not a fix!

[iham]

i tried the same thing (TTW) with the core-dev buildout branch 5.2. same result: broken.

[mauritsvanrees]

Confirmed.
IntIdMissingError is first raised and caught in plone.app.linkintegrity.utils.ensure_intid, where it tries to call intids.getId(obj) for the copy of the object. This fails, so it calls intids.register(obj) which succeeds.

It then fails in z3c.relationfield because a relation has self.from_object which is None:

[26]   /Users/maurits/shared-eggs/cp27m/plone.app.linkintegrity-3.3.9-py2.7.egg/plone/app/linkintegrity/handlers.py(110)modifiedContent()
-> updateReferences(obj, refs)
[27]   /Users/maurits/shared-eggs/cp27m/plone.app.linkintegrity-3.3.9-py2.7.egg/plone/app/linkintegrity/handlers.py(136)updateReferences()
-> _setRelation(obj, referencedRelationship, ref)
[28]   /Users/maurits/shared-eggs/cp27m/z3c.relationfield-0.8.0-py2.7.egg/z3c/relationfield/event.py(141)_setRelation()
-> catalog.index_doc(id, value)
[29]   /Users/maurits/shared-eggs/cp27m/zc.relation-1.1.post2-py2.7.egg/zc/relation/catalog.py(557)index_doc()
-> relToken, rel, value_index_info)
[30]   /Users/maurits/shared-eggs/cp27m/zc.relation-1.1.post2-py2.7.egg/zc/relation/catalog.py(486)_indexNew()
-> rel, value_index_info)
[31]   /Users/maurits/shared-eggs/cp27m/zc.relation-1.1.post2-py2.7.egg/zc/relation/catalog.py(588)_getValuesAndTokens()
-> values = getattr(valueSource, data['attrname'])
[32]   /Users/maurits/shared-eggs/cp27m/z3c.relationfield-0.8.0-py2.7.egg/z3c/relationfield/relation.py(33)from_id()
-> return intids.getId(self.from_object)

I guess this needs a fix in one of the functions called by plone.app.linkintegrity.handlers. modifiedContent.

[mauritsvanrees]

Looking a bit further, it goes wrong because a line in _setRelation does not work:

value.from_object = obj

Here, obj is the fresh copy of the page. But after this line when you ask what value.from_object is, the answer is None. This is because the from_object accessor tries to get the object via its intid, which fails with a KeyError, and results in None in relation.py.

[pbauer]

I think the problem may be that five.intid.keyreference.KeyReferenceToPersistent.wrapped_object does not catch the KeyError you mentioned but only NotFound and AttributeError when trying to traverse to the object that is set as value.from_object = obj in z3c.relationfield.event._setRelation. So value.from_object will be None instead of the unwrapped object and we end up with a IntIdMissingError.

I think that something in unrestrictedTraverse may have changed in Zope4 (I think I encountered that before somewhere) but I'd have to check with Plone 5.1.

When adding KeyError to the list of handled Exceptions in wrapped_object it works fine and the linkintegrity-relation is set up correctly, i.e. when trying to delete the image I get a warning from the original document and the from the copy.

[pbauer]

I fixed it in https://github.com/plone/five.intid/pull/12 and added a regression-test in https://github.com/plone/plone.app.linkintegrity/pull/71 that should pass once the fix is merged.

[pbauer]

https://github.com/plone/five.intid/pull/12 could use someone to look at investigating the two failing tests.

[jensens]

I debugged the failure in plone.app.linkintegrity:

Prior to the change in keyreference.py there was a KeyError raised. It was catched in https://github.com/zopefoundation/z3c.relationfield/blob/cdf764e1afb2a7cc0a67d572b7da828a11006700/src/z3c/relationfield/relation.py#L127
and so the returned value was None.

Now, correctly the object held as reference on the KeyReference itself is returned when the unrestrictedTraverse fails. This seems odd but is needed for cases, where objects are in creation and not connected to a folder (as far as I understand).

The failing plone.app.linkintegrity test is based on the assumption, that a broken reference has no objects any longer to return. In fact it still has this reference and returns it now.

Changing the test to accept it fixes it, like here https://github.com/plone/plone.app.linkintegrity/commit/010763aa1bf3bd3953a177a0557b3899e52f0477

I am not sure if this is the perfect soltion, but in my opinion this is enough, all other changes are fine.
if there is custom code around checking for broken references against None, this may break.

What do you think.

[pbauer]

lgtm

[jensens]

PR here https://github.com/plone/plone.app.linkintegrity/pull/72

[mauritsvanrees]

@jensens wrote:

Now, correctly the object held as reference on the KeyReference itself is returned when the unrestrictedTraverse fails. This seems odd but is needed for cases, where objects are in creation and not connected to a folder (as far as I understand).

That seemed odd to me as well, which made me hesitant on approving https://github.com/plone/five.intid/pull/12, but I guess it may be okay. It has worked so far, so apparently this special case is needed. Happy to have the PRs merged when tests pass.