Python-zeep: Error with an array in Request

Created on 1 Aug 2017 · 12Comments · Source: mvantellingen/python-zeep

When I created a Request that required an array of input messages, then I get the following error messages from the SOAP Server:

{
'status': {
'code': 3,
'message': 'Error in Filter: unknown field "" (field has to be part of datatype "InvoiceBaseFilterable")'
},
'result': None
}

I think the issue is similar to:

145

zeep version 2.2.0

WSDL File:
WSDL

Example Script:
```python

-- coding: utf-8 --

import datetime
from zeep import Client

"""
Array Error
"""

gsSoapPass = 'oJA9Jb5sGtfEHP0f03O0'
gsSoapClient = Client(
'https://zeep.gsales.info/api/api.php?wsdl&safedate=1',
)

year = 2017
gsFilterType = gsSoapClient.get_type('ns0:Filter')
gsFilterCustomerno = gsFilterType(
field='customerno',
operator='is',
value='5001',
)
gsFilterCreatedBigger = gsFilterType(
field='created',
operator='bigger',
value=datetime.datetime(year, 1, 1, 0, 0, 0, 0),
)
gsFilterCreatedSmaller = gsFilterType(
field='created',
operator='smaller',
value=datetime.datetime(year + 1, 1, 1, 0, 0, 0, 0),
)

No error Message, with passing only one filter in the Array

gsInvoicesCountYear = gsSoapClient.service'getInvoicesCount'
print(gsInvoicesCountYear)

Get error Message with more than one filter in the Array

gsInvoicesCountYear = gsSoapClient.service'getInvoicesCount'
print(gsInvoicesCountYear)
´´´

question

Source

TCRserver

All 12 comments

What steps did you take to debug this yourself? Did you compare the generated XML output?

mvantellingen on 6 Aug 2017

I have different zeep output variants tested with SoapUI.
For every variant I got the same error.

If I describe the same function with python suds, then a different XML output is generated.
And it works.

Apologize if I do not see the solution, but I'm not a professional in XML / Soap

Output from zeep

<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
    <soap-env:Body>
        <ns0:getInvoicesCount xmlns:ns0="gsales2">
            <apikey>oJA9Jb5sGtfEHP0f03O0</apikey>
            <filter>
                <Filter>
                    <field>customerno</field>
                    <operator>is</operator>
                    <value>5001</value>
                </Filter>
                <Filter>
                    <field>created</field>
                    <operator>bigger</operator>
                    <value>2017-01-01 00:00:00</value>
                </Filter>
                <Filter>
                    <field>created</field>
                    <operator>smaller</operator>
                    <value>2018-01-01 00:00:00</value>
                </Filter>
            </filter>
        </ns0:getInvoicesCount>
    </soap-env:Body>
</soap-env:Envelope>

Output from suds

<SOAP-ENV:Envelope xmlns:ns3="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns0="gsales2" xmlns:ns1="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns2="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ns4="gsales2api" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Header/>
<ns1:Body>
    <ns0:getInvoicesCount>
        <apikey xsi:type="ns2:string">oJA9Jb5sGtfEHP0f03O0</apikey>
        <filter xsi:type="ns4:ArrayOfFilter" ns3:arrayType="ns4:Filter[3]">
            <item xsi:type="ns4:Filter">
                <field xsi:type="ns2:string">customerno</field>
                <operator xsi:type="ns2:string">is</operator>
                <value xsi:type="ns2:string">5001</value>
            </item>
            <item xsi:type="ns4:Filter">
                <field xsi:type="ns2:string">created</field>
                <operator xsi:type="ns2:string">bigger</operator>
                <value xsi:type="ns2:string">2016-01-01 00:00:00</value>
            </item>
            <item xsi:type="ns4:Filter">
                <field xsi:type="ns2:string">created</field>
                <operator xsi:type="ns2:string">smaller</operator>
                <value xsi:type="ns2:string">2018-01-01 00:00:00</value>
            </item>
        </filter>
    </ns0:getInvoicesCount>
</ns1:Body></SOAP-ENV:Envelope>

TCRserver on 10 Aug 2017

Is the implementation that I have chosen with zeep correct,
or I have not understood something?

TCRserver on 10 Aug 2017

I'm having a similar problem. The relevant part of the wsdl

<xs:complexType name="TSecondaryDomains">
  <xs:annotation>
    <xs:documentation>a list of domain names</xs:documentation>
  </xs:annotation>
  <xs:complexContent>
    <xs:restriction base="soapenc:Array">
      <xs:attribute ref="soapenc:arrayType"
                   wsdl:arrayType="xs:string[]" />
    </xs:restriction>
  </xs:complexConent>
</xs:complexType>

And I'm building the structure with

secondary_domain_type = client.get_type('ns0:TSecondaryDomains')
domains = secondary_domain_type(['testdomain'])

The generated message contains

<SecondaryDomains>
  <item xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">testdomain</item>
</SecondaryDomains>

It seems to be missing the soapenc:arrayType='xsd:string[1]' which is required by the perl Soap::Lite service I'm calling. Soap::Lite gives the error:

aDomainNames is no array reference (Array) in call to REDACTED::setSecondaryDomains at /../site_perl/SOAP/Lite.pm line 2875

This works (manually crafted in postman):

<SecondaryDomains soapenc:arrayType='xsd:string[1]' xsi:type='soapenc:Array'>
  <item xsi:type='xsd:string'>testdomain</item>
</SecondaryDomains>

These test were done with zeep version 2.4.0

I can provide a minimal test script by email with required authentication information if that helps?

tswfi on 12 Dec 2017

Unfortunately the problem still exists. I am using the same WSDL as the issue creator and I have the exactly same behavior. I would be happy about any ideas :-)

mstrauch on 27 May 2018

To add more details:

PHP soapclient generates this XML output which is working:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="gsales2" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ns2="gsales2api" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  <SOAP-ENV:Body>
  <ns1:getInvoices>
    <apikey xsi:type="xsd:string">XXXXX</apikey>
    <filter SOAP-ENC:arrayType="ns2:Filter[2]" xsi:type="ns2:ArrayOfFilter">
      <item xsi:type="ns2:Filter">
        <field xsi:type="xsd:string">created</field>
        <operator xsi:type="xsd:string">bigger</operator>
        <value xsi:type="xsd:string">2018-01-01</value>
      </item>
      <item xsi:type="ns2:Filter">
        <field xsi:type="xsd:string">created</field>
        <operator xsi:type="xsd:string">smaller</operator>
        <value xsi:type="xsd:string">2018-01-31</value>
      </item>
    </filter>
    <sort xsi:type="ns2:Sort">
      <field xsi:type="xsd:string">invoiceno</field>
      <direction xsi:type="xsd:string">asc</direction>
    </sort>
    <recordcount xsi:type="xsd:integer">1</recordcount>
    <recordoffset xsi:type="xsd:integer">0</recordoffset>
  </ns1:getInvoices>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Output from zeep:

<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
  <soap-env:Body>
    <ns0:getInvoices xmlns:ns0="gsales2">
      <apikey>XXXXX</apikey>
      <filter>
        <Filter>
          <field>created</field>
          <operator>bigger</operator>
          <value>2018-01-01 00:00:00</value>
        </Filter>
        <Filter>
          <field>created</field>
          <operator>smaller</operator>
          <value>2018-01-30 00:00:00</value>
        </Filter>
      </filter>
      <sort>
        <field>invoiceno</field>
        <direction>asc</direction>
      </sort>
      <recordcount>1</recordcount>
      <recordoffset>0</recordoffset>
    </ns0:getInvoices>
  </soap-env:Body>
</soap-env:Envelope>

Error: Error in Filter: unknown field "" (field has to be part of datatype "InvoiceBaseFilterable")

mstrauch on 27 May 2018

@mstrauch I solved my problem with a filter. something like this

from zeep import Plugin

class ArrayFixer(Plugin):
    """ try to fix the soap arrays for Soap::Lite
    see https://github.com/mvantellingen/python-zeep/issues/521 
    TODO: There must be a better way 
    """

    def ingress(self, envelope, http_headers, operation):
        return envelope, http_headers

    def egress(self, envelope, http_headers, operation, binding_options):
        """ force array type to SecondaryDomains element """
        secondarydomains = envelope.find(".//SecondaryDomains")
        if secondarydomains is not None:
            logger.debug("Mangling secondarydomains element, adding arraytype")
            length = len(secondarydomains)
            secondarydomains.attrib[
                "{http://schemas.xmlsoap.org/soap/encoding/}arrayType"
            ] = "ns1:string[{}]".format(length)

            logger.debug("And removing xsitype from items")
            for item in secondarydomains.getchildren():
                item.attrib.clear()

        additional = envelope.find(".//AdditionalAttributes")
        if additional is not None:
            logger.debug("Mangling secondarydomains element, adding arraytype")
            length = len(additional)
            additional.attrib[
                "{http://schemas.xmlsoap.org/soap/encoding/}arrayType"
            ] = "ns1:anyType[{}]".format(length)

            logger.debug("And removing xsitype from items")
            for item in additional.getchildren():
                item.attrib.clear()

        return envelope, http_headers

And then when you are initializing the client add the plugin

        client = Client(
            wsdl=self.wsdl,
            strict=False,  
            transport=Transport(session=session),
            plugins=[arrayfixer, booleanfixer]
        )

Its not very pretty but works for me currently. (and just noticed a typo in my log statement :)

tswfi on 29 May 2018

😕1

Thanks for the hint :-) I can workaround my issue with your filter. Why does zeep generate wrong XML at this point?

mstrauch on 30 May 2018

🎉1

do we have a fix for Zeep ?

Namit-Jain-1987 on 20 May 2019

Hello, I'm facing the same issue.
In my case, the XML which I have to send to the service is:

<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
    <soap-env:sepaOperations>
      <providerID>33</providerID>
      <operations>
        <item>
          <operationType>3</operationType>
          <merchantCode>uwhwe33</merchantCode>
          <terminalIDDebtor>9076</terminalIDDebtor>
          <uniqueIdCreditor>76543</uniqueIdCreditor>
          <companyNameCreditor>Test</companyNameCreditor>
          <ibanNumberCreditor>IBANACCOUNT</ibanNumberCreditor>
          <swiftCodeCreditor></swiftCodeCreditor>
          <companyTypeCreditor>4</companyTypeCreditor>
          <operationOrder>9889087</operationOrder>
          <operationAmount>40668</operationAmount>
          <operationCurrency>EUR</operationCurrency>
          <operationDatetime>20190620</operationDatetime>
          <operationConcept>Test concept</operationConcept
          <operationSignature>CACF59FDB72C9EB5A98C9E69</operationSignature>
        </item>
      </operations>
    </soap-env:sepaOperations>
  </soap-env:Body>
</soap-env:Envelope>

However, zeep generates this request:

<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
  <soap-env:Body>
    <soap-env:sepaOperations>
      <providerID>33</providerID>
      <operations>
        <operation>
          <operationType>3</operationType>
          <merchantCode>uwhwe33</merchantCode>
          <terminalIDDebtor>9076</terminalIDDebtor>
          <uniqueIdCreditor>76543</uniqueIdCreditor>
          <companyNameCreditor>Test</companyNameCreditor>
          <ibanNumberCreditor>IBANACCOUNT</ibanNumberCreditor>
          <swiftCodeCreditor></swiftCodeCreditor>
          <companyTypeCreditor>4</companyTypeCreditor>
          <operationOrder>9889087</operationOrder>
          <operationAmount>40668</operationAmount>
          <operationCurrency>EUR</operationCurrency>
          <operationDatetime>20190620</operationDatetime>
          <operationConcept>Test concept</operationConcept
          <operationSignature>CACF59FDB72C9EB5A98C9E69</operationSignature>
        </operation>
      </operations>
    </soap-env:sepaOperations>
  </soap-env:Body>
</soap-env:Envelope>

Note that element "item" is changed by "operation", and it makes the request fail. I will try to apply tswfi's workaround but it would be great "cleaner" way is available.

Anyway, congrats for the library! it is very helpful.
Regards.

iubago on 11 Jun 2019

Running into the same issue here at well. Will give the workaround a try.

mmendenhall on 27 May 2020

Ran into this issue and fixed it by overriding the Transport class and modifying the raw request. While using the Plugin system is a more palatable approach, it'd be great if there was official support in the library for this use case.