Lizmap-web-client: Compatibility with QGIS3

[mdouchin]

This issue aims to list all the Lizmap features to check agains QGIS 3.4 which will be next LTR in the end of February 2019. I would like anyone interested to contribute to the list :

• please add a comment with missing items (and I would add them in the main issue description)
• please check the feature and report if there is an issue. We will open a new Github issue and reference it here when needed

Landing page

• [x] Lizmap landing page aka "Projects" page shows all the QGIS projects configured with Lizmap plugin (with a *.cfg file)

Map options

• [x] Lizmap shows the layer data in the map and respects the extent

Layer tree

• [x] The legend shows all QGIS layers
• [x] The legend tree is like the one in QGIS
• [ ] The legend tree shows the embed layers
• [x] For each layer, the legend image is displayed
• [x] For each layer, the legend image respects the scale and symbology defined in QGIS
• [x] For each layer, the legend image allows to switch between different styles as defined in QGIS

Rendering

• [x] The layers are rendered in the map Only with QGIS >= 3.4.5
• [x] The layers are rendered only between the defined scales when they are set
• [x] The layers are rendered even if their projection differs from the Lizmap map display projection
• [x] The layers are rendered in the correct order based on their position in legend
• [x] The layers are rendered in the correct order based on their position in legend when the order has been changed in QGIS with the dedicated dock panel
• [x] The rendering of a group of layers is correct and respects the layers order within the group
• [x] The image type (ex: jpeg) is respected

Print

• [x] The composer layouts are shown in the print tool
• [x] The exported map respects QGIS layout
• [x] The different formats work : PDF, SVG, etc.

Popup

• [x] Popups are shown for configured layers
• [x] Popup auto mode shows the correct values, even for value relation or value list fields
• [x] Popup content respects the QGIS tooltip configuration when used, with various expressions
• [x] Popup content respects the Lizmap HTML configuration when used
• [x] Tolerance for points, lines and polygons are respected

Locate by layer

Attribute table

Selection and filter

Editing

• [x] Display an existing PgSQL feature (without geometry)
• [x] Create a PgSQL feature (without geometry)
• [x] Delete a PgSQL feature (without geometry)
• [x] Modify a PgSQL feature (without geometry)
• [x] Display an existing PgSQL feature (with geometry)
• [x] Create a PgSQL feature (with geometry)
• [x] Delete a PgSQL feature (with geometry)
• [x] Modify a PgSQL feature (with geometry)

Tooltip

Filtering data

Dataviz

Time manager

Atlas

WMS and WMTS external layers

Here is a list of contributors (not exhaustive)
@rldhont @mbernasocchi @pcav @laurentj @Gustry @slarosa @dmarteau @josemvm @nboisteault @vherreros @gerald2545 @mathieubossaert @majorxtrem @jankoja @GabrielePrestifilippo @nliaudat @jancelin @jaitor1 @jonnyforestGIS

List of available testers for early version (please tell us when and which branch is suitable for testing)
@pcav

[mdouchin]

@pcav @rldhont @Gustry @laurentj @dmarteau beware that we cannot edit the issue main description concurrently. Has anyone an idea of how to avoid loosing some data while another is editing the issue ? Should we only use comments to add items (and I will be in charge of copying them in the main description) or is there another preffered way ?

[pcav]

Sorry, I was not aware of this. Comments are OK for me, I'll refrain from direct editing. Hope nothing has been really lost.

[mbernasocchi]

@mdouchin great!

• locate by layer seems not to work, but I'havent tested thoroughly
• main.css in the default custom theme is ignored on the landing page (but I'm not sure if this is supposed to work at all)

@signedav @marioba have more feedback?

[rldhont]

Linked open issues:

• ~The print icon disappears in QGIS 3 #1100~
• ~QGIS Server 3.4.4 does not show the map anymore #1059~
• ~child objets in popup with QGIS server 3 #1024~
• ~QGIS server 3 #1018~
• ~Composer layouts not recognized at all when using QGIS3 #1007~
• ~QGIS Server 3: Can't correctly print or use the selection tool #971~
• ~lizmap 3 support for QGIS 3 desktop and server #857~

[pcav]

Given that QGIS 2.18 now seems essentially unavailable from repos, I think this issue is of absolute importance.

[pcav]

1059 IMHO should be closed

[josemvm]

attribute table not avaiable for many many layers stored in postgreSQL

[rldhont]

@gioman can you be more precise ? Which QGIS Server version ?

[josemvm]

attribute table not avaiable for many many layers stored in postgreSQL

qgis server 3.4.5

[rldhont]

@josemvm do you have a link to test it ?

[gioman]

@gioman can you be more precise ? Which QGIS Server version ?

@rldhont I'm not part of this conversation :)

[rldhont]

sorry @gioman

[josemvm]

@rldhont please check your mailbox

[rldhont]

@josemvm it seems there is an issue when BBOX is added to the GetFeature Request. Can you open an issue about AttributeTable with extent restriction.

[josemvm]

attribute table not avaiable for many many layers stored in postgreSQL

if "limitDataToBbox": "True" is set in the plugin https://github.com/3liz/lizmap-web-client/issues/1084#issuecomment-467580894

[josemvm]

edition - all the widgets are not taken by the qgis server

[nboisteault]

Landing page for me is ok.

[nboisteault]

@mbernasocchi for me locate by layer works with a QGIS 2.18 project and QGIS Server 3.4.5. We have to test with a QGIS 3.4.5 project .

[josemvm]

@nboisteault, qgis server 3.4.5 also prints 2.14 projects

[mbernasocchi]

@mbernasocchi for me locate by layer works with a QGIS 2.18 project and QGIS Server 3.4.5. We have to test with a QGIS 3.4.5 project .

@nboisteault you're right, it was a mistake on our side (PGSERVICE file missing in docker), we didn't know that lizmap does direct queries to the DB. why is that?

[rldhont]

@mbernasocchi lizmap is requesting directly data from Postgis instead of QGIS Server WFS GetFeature for faster request. But if lizmap cannot request Postgis, it does the WFS Request.

[mbernasocchi]

instead of QGIS Server WFS GetFeature for faster request.

it is what we immagined

But if lizmap cannot request Postgis, it does the WFS Request.

then this is the part that was not working for us.

[nboisteault]

attribute table not avaiable for many many layers stored in postgreSQL

@josemvm You said "many layers". Do you watch differences between those working and those not ?

[josemvm]

@nboisteault, no, the problem only happens when "limitDataToBbox": "True" is set in the lwc plugin

[nboisteault]

edition - all the widgets are not taken by the qgis server

@josemvm which ones please ?

[josemvm]

hi @nboisteault i'm talking about projects saved with qgis server 3.4.5

[jonnyforestGIS]

@rldhont , @mdouchin do you define specific test data for this testing? Can we use the demo projects montepellier?

[rldhont]

@jonnyforestGIS we use the demo project montpellier and some specific if needed

[jonnyforestGIS]

@rldhont ok I will see the checklist above and will give feedback about unchecked items

[jonnyforestGIS]

@rldhont @mdouchin
Test Printing Section:
OS: Windows 10 Home 64-bits
QGIS Server 3.4.5 LTR 64-bits
php7.2.16 64-bits
Apache 2.4 64-bits
Lizmap Web-client-master
Lizmap Project - Montpellier - Transports - Demo

PDF 100,200,300dpi with labels description A4 Landscape - ✔️
PNG 100,200,300dpi with labels description A4 Landscape - ✔️
JPG 100,200,300dpi with labels description A4 Landscape - ❌ Error:

 <ServiceExceptionReport xmlns="http://www.opengis.net/ogc" version="1.3.0" capture-installed="true"> <ServiceException code="InvalidFormat"> Output format jpg is not supported by the GetPrint request </ServiceException> </ServiceExceptionReport>

SVG 100,200,300dpi with labels description A4 Landscape - ❌ Error: Give output map but with errors:

This page contains the following errors: error on line 374 at column 11: Namespace prefix rdf on RDF is not defined error on line 375 at column 12: Namespace prefix cc on Work is not defined error on line 376 at column 15: Namespace prefix dc on format is not defined error on line 377 at column 14: Namespace prefix dc on title is not defined error on line 378 at column 13: Namespace prefix dc on date is not defined error on line 379 at column 19: Namespace prefix dc on identifier is not defined error on line 380 at column 20: Namespace prefix dc on description is not defined error on line 381 at column 16: Namespace prefix dc on creator is not defined error on line 382 at column 15: Namespace prefix cc on Agent is not defined error on line 383 at column 16: Namespace prefix dc on title is not defined error on line 386 at column 18: Namespace prefix dc on publisher is not defined error on line 387 at column 15: Namespace prefix cc on Agent is not defined error on line 388 at column 16: Namespace prefix dc on title is not defined error on line 392 at column 14: Namespace prefix rdf on Bag is not defined Below is a rendering of the page up to the first error.

[jonnyforestGIS]

Save project Montpellier - Transports - Demo as project .qgs in QGIS 3.4.5 Desktop LTR and the module of printing disappears.
Confirm behaviour. Making changes in the layout and save the project the print module don't appears anymore equal #1100

[rldhont]

@jonnyforestGIS I can confirm

[nboisteault]

I can confirm issue with jpg format. I found the error message here. Doc says that jpeg is supported though. @rldhont Issue reported here.

[jonnyforestGIS]

So probably I will ask a newbie question :smile: for the problem jpg. I'm not the c++ developer but make some sense. If we use the extra else if will work?
Something like this:

else if ( format == QgsWmsParameters::JPG )
    {
      contentType = "application/jpg";
    }

But in the other hand i search in the section of the file qgswmsutils.cpp
and in this file refers to jpeg

[rldhont]

the message comes from here https://github.com/qgis/QGIS/blob/master/src/server/services/wms/qgswmsrenderer.cpp#L588

I don't know why this restriction, I need to take a look at the 2.18 server code.

[rldhont]

It seems that the solution is to update https://github.com/qgis/QGIS/blob/master/src/server/services/wms/qgswmsrenderer.cpp#L517 by added

|| format == QgsWmsParameters::JPEG

And added to https://github.com/qgis/QGIS/blob/master/src/server/services/wms/qgswmsgetprint.cpp#L41

    else if ( format == QgsWmsParameters::JPEG )
    {
      contentType = "image/jpeg";
    }

[rldhont]

Some QGIS PR done https://github.com/qgis/QGIS/pull/9525

[Gustry]

Thanks for the fix @rldhont
So you can tick the JPEG box? (last one in rendering section)

Are there still some other "big" blockers? I noticed a lot of commits these last 2 weeks. Do you have a estimated ETA for Lizmap 3.3? QGIS 3.4.6 has been released last week.

Sorry I have a limited time here at work to try QGIS Server 3. But I would like to migrate my current organization with QGIS 3.

[josemvm]

print selection https://github.com/3liz/lizmap-web-client/issues/1153

[rmarzocchi84]

@mbernasocchi for me locate by layer works with a QGIS 2.18 project and QGIS Server 3.4.5. We have to test with a QGIS 3.4.5 project .

@nboisteault you're right, it was a mistake on our side (PGSERVICE file missing in docker), we didn't know that lizmap does direct queries to the DB. why is that?

I do not understand what do you mean with "pgservice file" but using QGIS server 3.4.6 and last lizmap locate by layer function do not works for me

I think it can be a problem with WFS because a similar problem affect download of layer and table view functions.

[rldhont]

@rmarzocchi84 have you some errors in logs ?

[rmarzocchi84]

Yes but not easily understandable

2019-04-01 16:36:07 93.149.193.29 error 2019-04-01 16:36:07 XXXX.XX.XX [0] The QGIS project /home/gter/nextcloud-data/$
/lizmap-web-client/lizmap/www/index.php/view/
array ( 'module' => 'view', 'action' => 'default:index', )

0 lizmapProject->__construct() /home/gter/lizmap-web-client-3.3git/lizmap/modules/lizmap/classes/lizmap.class.php : 298
1 lizmap::getProject() /home/gter/lizmap-web-client-3.3git/lizmap/modules/lizmap/classes/lizmapRepository.class.php : 150
2 lizmapRepository->getProjects() /home/gter/lizmap-web-client-3.3git/lizmap/modules/view/zones/main_view.zone.php : 47
3 main_viewZone->_prepareTpl() /home/gter/lizmap-web-client-3.3git/lib/jelix/utils/jZone.class.php : 239
4 jZone->_createContent() /home/gter/lizmap-web-client-3.3git/lib/jelix/utils/jZone.class.php : 206
5 jZone->getContent() /home/gter/lizmap-web-client-3.3git/lib/jelix/utils/jZone.class.php : 294
6 jZone::_callZone() /home/gter/lizmap-web-client-3.3git/lib/jelix/utils/jZone.class.php : 95
7 jZone::get() /home/gter/lizmap-web-client-3.3git/lib/jelix/tpl/jTpl.class.php : 125
8 jTpl->assignZone() /home/gter/lizmap-web-client-3.3git/lizmap/modules/view/controllers/default.classic.php : 101
9 defaultCtrl->index() /home/gter/lizmap-web-client-3.3git/lib/jelix/core/jCoordinator.class.php : 225
10 jCoordinator->process() /home/gter/lizmap-web-client-3.3git/lizmap/www/index.php : 23
`

[rmarzocchi84]

@rldhont sorry probably is an error of apache. I change the level of log of QGIS from 1 to 2 to view better the error and restart apache.
Now locate by layer works correctly (also table view and export)

I will perform more test for debug and inform you! Thanks for all your work

[rmarzocchi84]

There is a problem when max and min zoom level for map is set up.

Already test in qgis-server 3.4.6 and it is not a problem of WMS which is corrected represented

Probably it is due to a strange nomenclature of the WMS capabilities where MinScaleDenominator is the "maximum scale":

<Layer queryable="1">
<Name>palazzi_rolli</Name>
<Title>palazzi_rolli</Title>
<CRS>CRS:84</CRS>
<CRS>EPSG:3857</CRS>
<EX_GeographicBoundingBox>
<westBoundLongitude>8.92508</westBoundLongitude>
<eastBoundLongitude>8.93759</eastBoundLongitude>
<southBoundLatitude>44.4058</southBoundLatitude>
<northBoundLatitude>44.4157</northBoundLatitude>
</EX_GeographicBoundingBox>
<BoundingBox miny="5.52846e+6" minx="993535" CRS="EPSG:3857" maxx="994928" maxy="5.53e+6"/>
<Style>
<Name>predefinito</Name>
<Title>predefinito</Title>
<LegendURL>
<Format>image/png</Format>
<OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink" xlink:type="simple" xlink:href="https://www.gishosting.gter.it/lizmap-web-client/lizmap/www/index.php/lizmap/service/?repository=workshop&project=progetto_test&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetLegendGraphic&LAYER=palazzi_rolli&FORMAT=image/png&STYLE=predefinito&SLD_VERSION=1.1.0"/>
</LegendURL>
</Style>
<MinScaleDenominator>0</MinScaleDenominator>
<MaxScaleDenominator>25000</MaxScaleDenominator>
</Layer>

[rldhont]

@rmarzocchi84 a new version of the plugin has been released to provide right min/max scale.

• release https://github.com/3liz/lizmap-plugin/releases/tag/3.0.2
• fix https://github.com/3liz/lizmap-plugin/issues/122

[rmarzocchi84]

Ok I tested it and I can confirm it works @rldhont

[pcav]

Do you have a tentative date for string freeze?
This would be useful to schedule translation timing.
Thanks.

[rmarzocchi84]

edition - all the widgets are not taken by the qgis server

Problem confirmed or it is a mistake in my configuration? If confirmed I will try to understand the reason

[rmarzocchi84]

Effectively the .qgs file are really different:

e.g.

qgis 2.18.28

    <edittype widgetv2type="ValueMap" name="visitabile">
      <widgetv2config fieldEditable="1" constraint="" labelOnTop="0" constraintDescription="" notNull="1">
        <value key="No" value="f"/>
        <value key="Sì" value="t"/>
      </widgetv2config>
    </edittype>

qgis 3.4.6

    <field name="visitabile">
      <editWidget type="ValueMap">
        <config>
          <Option type="Map">
            <Option name="map" type="List">
              <Option type="Map">
                <Option value="t" name="Sì" type="QString"/>
              </Option>
              <Option type="Map">
                <Option value="f" name="No" type="QString"/>
              </Option>
            </Option>
          </Option>
        </config>
      </editWidget>
    </field>

So it is really difficult to guarantee the compatibility for both the projects also in lizmap

[mdouchin]

This is a blocker. We cannot release 3.3 before this is properly fixed. There are already code inside Lizmap to manage different versions of QGIS, since the edit widgets properties have already change in the past.

Example here: https://github.com/3liz/lizmap-web-client/blob/master/lizmap/modules/lizmap/classes/qgisFormControl.class.php#L235
We should properly handle this for this bigger change. @laurentj has ideas about this

[rldhont]

@rmarzocchi84 I have created a PR to fix edition config from QGIS Project 3, can you test it ?

[rmarzocchi84]

@rldhont there is a specific branch to test or I update the master edition of lizmap?

[rmarzocchi84]

Ok @rldhont I found the PR in your repository (sorry for the previous question)

From a preliminary test it seems to work correctly.

First of all I test:

• Value Map
• Datetime field

I only find a strange behaviour that perhaps depends on a QGIS bug:

This value map seems not correctly defined

<Option name="map" type="Map">

and do not works:

<field name="d_origine">
      <editWidget type="ValueMap">
        <config>
          <Option type="Map">
            <Option name="map" type="Map">
              <Option name="Altro" value="5" type="QString"/>
              <Option name="Atti amministrativi" value="3" type="QString"/>
              <Option name="Palmare" value="4" type="QString"/>
              <Option name="Rilievo manuale" value="1" type="QString"/>
              <Option name="Rilievo meccanico" value="2" type="QString"/>
            </Option>
          </Option>
        </config>
      </editWidget>
    </field>

This value map is correctly defined

<Option name="map" type="List">

and works:

    <field name="d_origine">
      <editWidget type="ValueMap">
        <config>
          <Option type="Map">
            <Option name="map" type="List">
              <Option type="Map">
                <Option name="Altro" value="5" type="QString"/>
              </Option>
              <Option type="Map">
                <Option name="Atti amministrativi" value="3" type="QString"/>
              </Option>
              <Option type="Map">
                <Option name="Palmare" value="4" type="QString"/>
              </Option>
              <Option type="Map">
                <Option name="Rilievo manuale" value="1" type="QString"/>
              </Option>
              <Option type="Map">
                <Option name="Rilievo meccanico" value="2" type="QString"/>
              </Option>
            </Option>
          </Option>
        </config>
      </editWidget>
    </field>

After I test Value Relation and it do not works. This is the QGIS project definition:

    <field name="cod_strada">
      <editWidget type="ValueRelation">
        <config>
          <Option type="Map">
            <Option name="AllowMulti" value="false" type="bool"/>
            <Option name="AllowNull" value="false" type="bool"/>
            <Option name="FilterExpression" value="" type="QString"/>
            <Option name="Key" value="cod_strada" type="QString"/>
            <Option name="Layer" value="route20190320155525256" type="QString"/>
            <Option name="NofColumns" value="1" type="int"/>
            <Option name="OrderByValue" value="true" type="bool"/>
            <Option name="UseCompleter" value="true" type="bool"/>
            <Option name="Value" value="cod_strada" type="QString"/>
          </Option>
        </config>
      </editWidget>
    </field>

[rldhont]

Thanks for the review, I'll update the code.

[nboisteault]

@rldhont I tried QGIS Constraints "Not null" and "Enforce not null constraint", both didn't work and didn't show field as mandatory.

[nboisteault]

I have tried with QGIS 2.18 and it is the same behavior so it is not a regression but an needed enhancement.

[josemvm]

until now, constraints like "not null" or "default value" only could be set at the postgresql level

[pcav]

AFAIK version 3.2.3 should be compatible with QGIS server 3, correct? I started tessting it, and I noticed that printing is missing: is this expected?
Thanks.

[Gustry]

@pcav It's LWC 3.3 which will be fully compatible with QGIS 3.4, not LWC 3.2.

You should try https://github.com/3liz/lizmap-web-client/releases/tag/3.3rc1

[pcav]

Thanks, will do.

[rldhont]

@pcav you have to change the qgisServerVersion in the backoffice. If the QGIS project is a 3., to have print capabilities, the QGIS Server version has to be 3.. But the layout description is parsed in LWC 3.3.

If you have some trouble with scales: there was an issue in the plugin https://github.com/3liz/lizmap-plugin/issues/122 and a fix https://github.com/3liz/lizmap-plugin/pull/129

[pcav]

Thanks, I realized this too late. It would be good if somehow LM could check the QGIS version automatically, e.g. during installation, but that's obviously low priority.
Thanks a lot.

[majorxtrem]

"The layers are rendered in the correct order based on their position in legend when the order has been changed in QGIS with the dedicated dock panel" change 10 days ago

Is not working for me, any idea ?

Server : qgis-server 3.4.10+15buster
Lizmap : lizmap-web-client git master (on 27/07/19)
Client : Qgis 3.4.10 LTR
Qgis Plugin : lizmap 3.0.3

[nboisteault]

Confirmed. Debugging now.

[nboisteault]

@rldhont @mdouchin Does lizmap handle "Layer order" set in QGIS like in the screenshot below.

[rldhont]

It seems that not null constraints from QGIS Project 3 are not well detected by lizmap

[rldhont]

Constraint examples:

      <constraints>
        <constraint constraints="3" unique_strength="1" field="id" notnull_strength="1" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="objectid" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="commune" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="1" unique_strength="0" field="nom" notnull_strength="1" exp_strength="0"/>
        <constraint constraints="1" unique_strength="0" field="type" notnull_strength="1" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="statut" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="code_insee" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="ss_type" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="gestion" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="photo" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="valide" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="contact_te" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="courriel" notnull_strength="0" exp_strength="0"/>
        <constraint constraints="0" unique_strength="0" field="adresse" notnull_strength="0" exp_strength="0"/>
      </constraints>

[laurentj]

Now that the support of Qgis3 has been developed and released into lizmap 3.3, I think we should close this issue, and open new ones for new bugs related to the support of Qgis3.

[pcav]

+1