Che: Projects with "type": "zip" are extracted to a subpath, breaking devfiles

Created on 7 Oct 2019 · 10Comments · Source: eclipse/che

Describe the bug

Since https://github.com/eclipse/che/issues/14206 is resolved, Theia can handle projects downloaded as zips. However, typically zips (e.g. from github) unzip to a folder including the branch name, which causes the project to be unzipped to a subpath -- e.g.

/projects/console-java-simple/console-java-simple-master

instead of

/projects/console-java-simple/

This breaks many devfile commands that depend on the project being in a known location.

Che version

[x] nightly

Steps to reproduce

Start a workspace using the devfile

metadata:
  name: zip-project
projects:
  - name: console-java-simple
    source:
      location: 'https://github.com/che-samples/console-java-simple/archive/master.zip'
      type: zip
components:
  - id: redhat/java11/latest
    type: chePlugin
  - mountSources: true
    memoryLimit: 512Mi
    type: dockerimage
    volumes:
      - name: m2
        containerPath: /home/user/.m2
    image: 'quay.io/eclipse/che-java11-maven:nightly'
    alias: maven
    env:
      - value: /home/user/.m2
        name: MAVEN_CONFIG
      - value: >-
          -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
          -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
          -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true
          -Xms20m -Djava.security.egd=file:/dev/./urandom -Duser.home=/home/user
        name: MAVEN_OPTS
      - value: >-
          -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
          -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
          -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true
          -Xms20m -Djava.security.egd=file:/dev/./urandom
        name: JAVA_OPTS
      - value: >-
          -XX:MaxRAMPercentage=50 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
          -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4
          -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true
          -Xms20m -Djava.security.egd=file:/dev/./urandom
        name: JAVA_TOOL_OPTIONS
apiVersion: 1.0.0
commands:
  - name: maven build
    actions:
      - workdir: '${CHE_PROJECTS_ROOT}/console-java-simple'
        type: exec
        command: mvn clean install
        component: maven
  - name: maven build and run
    actions:
      - workdir: '${CHE_PROJECTS_ROOT}/console-java-simple'
        type: exec
        command: mvn clean install && java -jar ./target/*.jar
        component: maven

Expected behavior

Project is unzipped to the correct folder.

Runtime

[x] OpenShift 3.11

Screenshots

Screenshot from 2019-10-07 15-59-54

Additional Context

Impacts https://github.com/eclipse/che/issues/14733

areeditoche-theia kinbug severitP1 teaeditors

Source

amisevsk

Most helpful comment

@amisevsk @ibuziuk you can use git archive instead of zip.

l0rd on 23 Oct 2019

👍3

All 10 comments

@amisevsk I am setting severity/p2 because I suppose that's not a blocker for #14733. If, instead, it is a blocker let's flag it as severity/blocker.

l0rd on 8 Oct 2019

@l0rd :+1: it's probably not a blocker but also isn't the ideal flow; #14733 can be completed without issue, but commands in "offline" devfiles likely won't work until this issue is resolved.

amisevsk on 8 Oct 2019

@amisevsk not a blocker but I consider it a major issue if the commands don't work.

Why do you consider using the branch name as prefixing a standard? For instance if the zip is created using git archive -o <filename>.zip HEAD files are not prefixed. Instead of fixing it Theia side what about fixing how we generate the zip file?

l0rd on 8 Oct 2019

Setting severity/P1

l0rd on 8 Oct 2019

@l0rd I suppose the issue is that we're currently not generating the zip file and instead getting it directly from github. If you download and extract the zip for Che master, it'll be in the <repo>-<branchname> form. I can update the PR to instead do a git checkout and zip that, but it's a fair bit of added complexity.

amisevsk on 9 Oct 2019

@amisevsk knowing this behavior, should not the devfiles be created with the *-branch project name in mind. Also, where currently the zip dependent versions of devfiles are used? Could you please clarify why this already might be critical for the airgap scenario?

ibuziuk on 21 Oct 2019

@ibuziuk The zipped projects are used when you build an 'offline' devfile registry: --target offline-registry and the script

The two 'easy' fixes available would be

Rewrite devfiles in the offline build to be named *-branch, which is hard since commands would need to be updated as well
Change offline build to git clone and then zip projects in the directory, which is a bit messy but probably worthwhile

amisevsk on 21 Oct 2019

@amisevsk @l0rd

Obviously, factory plugin can't skip that first level directory for each zip project.
We could somehow analyze the content of an archive or it's location and decide then whether we need to skip that folder or not. But this way is error prone.
Personally I would like to have a new field for a project section in devfile which shall indicate that first level folder should be skipped.
Do you have any suggestions on this?